Most of software development is, in fact, communication. Even more so for free software projects that involve more than one person. Communication is an overhead, something you need to do on top of coding, so many hackers don't pay much attention to it. This is a mistake. A project whose members communicate effective gets more done in less time than one whose members don't. Here are some hints that may be useful:

• use the appropriate medium: if you need an answer soon, ask on a chat system such as IRC; if you can wait a while, use email or a blog post or a web forum

• be short and clear: a long, rambling question takes more time and effort to read, never mind respond to, and is often less clear and thus results in a less helpful answer

• take responsibility of getting the problem solved: develop a way to reproduce the problem, and if it's code, with the shortest, simplest piece of self-standing code you can (you'll often find you find the answer yourself)

• make it easy to help you: explain what you really want to achieve (not something else, even if you think it's easier), and what you've done, and what the exact result is (use copy-paste or take a screenshot)

• don't be insulting, arrogant, dismissive, or aggressive: you need help, don't make those you want it from not like you, or they might not even try

• say thank you: you'll be remembered well, and those who helped you (or tried to) will have more fun and are more motivated to work for free for others in the future

Before everyone had a multitude of computers of their own computers were rare, and if you wanted to use one you had to share it.

Given the demand for computers exceeded the supply, people had to share time using it.

Initially you could do this with a stop watch, but it's better for the computer itself to be able to measure this time since as computers became more complicated:

1. Pre-emption, where one process can be interrupted to run another, means the time take up by a program isn't just the difference between when the program started and ended.

2. Multi-threading, where a program can run multiple commands simultaneously, means you can use CPU time at a rate of more than one CPU second per second.

Computers became so pervasive that most computer users don't need to share, but virtual server providers also need to account for time used, and CPU time can also be used to measure how long it takes to perform an operation for profiling purposes so when a program is slow you know which part is the most worth your time to optimise.

Getting CPU time

The CPU time is read in the same way as other clocks, with different clock IDs for each process or thread.

1. Current process with CLOCK_PROCESS_CPUTIME_ID.

   int ret = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &time);
   

2. Current thread with CLOCK_THREAD_CPUTIME_ID.

   int ret = clock_gettime(CLOCK_THREAD_CPUTIME_ID, &time);
   

3. Another process with clock_getcpuclockid(3).

   int pid_gettime(pid_t pid, struct timespec *tp) {
       int ret;
       clockid_t clockid;
       ret = clock_getcpuclockid(pid, &clockid);
       if (ret != 0) {
           return ret;
       }
       ret = clock_gettime(clockid, tp);
       return ret;
   }
   

4. Another thread with pthread_getcpuclockid(3).

   int thread_gettime(pthread_t thread, struct timespec *tp) {
       int ret;
       clockid_t clockid;
       ret = pthread_getcpuclockid(thread, &clockid);
       if (ret != 0) {
           return ret;
       }
       ret = clock_gettime(clockid, tp);
       return ret;
   }
   

See gettime.c for an example program for reading the times, and Makefile for build instructions.

Profiling

We can instrument code (see profile-unthreaded.c) to see how much time a section takes to run.

#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h>

static void print_time(FILE *f, struct timespec time) {
    fprintf(f, "%lld.%09lld\n", (long long)time.tv_sec, (long long)time.tv_nsec);
}

int main(int argc, char **argv) {
    enum {
        ITERATIONS = 1000000,
    };
    int ret, exit = 0;
    struct timespec start, end;

    ret = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &start);
    if (ret != 0) {
        perror("clock_gettime");
        exit = 1;
        goto exit;
    }

    for (int i = 0; i < ITERATIONS; i++) {
        fprintf(stdout, "% 7d\n", i);
    }

    ret = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &end);
    if (ret != 0) {
        perror("clock_gettime");
        exit = 1;
        goto exit;
    }

    end.tv_sec -= start.tv_sec;
    end.tv_nsec -= start.tv_nsec;
    if (end.tv_nsec < 0) {
        end.tv_sec--;
        end.tv_sec += 1000000000l;
    }

    print_time(stderr, end);

exit:
    return exit;
}

$ make profile-unthreaded
$ ./profile-unthreaded >/tmp/f
0.073965395

We can make use of threads to try to speed this up (see profile-threaded.c).

#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h>
#include <unistd.h>

#define ARRAY_SIZE(x) (sizeof(x) / sizeof(*x))

static void print_time(FILE *f, struct timespec time) {
    fprintf(f, "%lld.%09lld\n", (long long)time.tv_sec, (long long)time.tv_nsec);
}

struct thread_args {
    int fd;
    int start;
    unsigned len;
};

void *thread_run(void *_thread_args) {
    struct thread_args *thread_args = _thread_args;
    char buf[9];
    for (int i = thread_args->start;
         i < thread_args->start + thread_args->len; i++) {
        ssize_t len = snprintf(buf, ARRAY_SIZE(buf), "% 7d\n", i);
        pwrite(thread_args->fd, buf, len, i * len);
    }
    return NULL;
}

int main(int argc, char **argv) {
    enum {
        ITERATIONS = 1000000,
        THREADS = 4,
    };
    int i, ret, exit = 0;
    struct timespec start, end;
    pthread_t threads[THREADS];
    struct thread_args thread_args[THREADS];

    ret = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &start);
    if (ret != 0) {
        perror("clock_gettime");
        exit = 1;
        goto exit;
    }

    for (i = 0; i < ARRAY_SIZE(threads); i++) {
        thread_args[i].fd = 1;
        thread_args[i].start = ITERATIONS / THREADS * i;
        thread_args[i].len = ITERATIONS / THREADS;
        ret = pthread_create(&threads[i], NULL, thread_run,
                             &thread_args[i]);
        if (ret != 0) {
            perror("pthread_create");
            exit = 1;
        }
    }
    if (exit != 0) {
        for (; i >= 0; i--) {
            (void) pthread_cancel(threads[i]);
        }
        goto exit;
    }
    
    for (i = 0; i < ARRAY_SIZE(threads); i++) {
        ret = pthread_join(threads[i], NULL);
        if (ret != 0) {
            perror("pthread_join");
            exit = 1;
        }
    }
    if (exit != 0) {
        goto exit;
    }

    ret = clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &end);
    if (ret != 0) {
        perror("clock_gettime");
        exit = 1;
        goto exit;
    }

    end.tv_sec -= start.tv_sec;
    end.tv_nsec -= start.tv_nsec;
    if (end.tv_nsec < 0) {
        end.tv_sec--;
        end.tv_sec += 1000000000l;
    }

    print_time(stderr, end);

exit:
    return 0;
}

$ make profile-threaded
$ ./profile-threaded >/tmp/f
3.185380729

By instrumenting we can tell that this actually made this section a lot slower.

Don't do this

Manually instrumenting things is a lot of work which means you are only going to do it for bits you already suspect are slow.

GCC's -pg adds instrumentation to dump times in a format readable by gprof.

valgrind when invoked like valgrind \-\-tool=callgrind prog and kcachegrind to view it. This runs your program on an emulated CPU, so it can use its own model of how long an operation takes for accounting time so it is unaffected by the overhead of profiling.

perf makes use of CPU features to measure with minimum overhead.

make CFLAGS=-ggdb command
perf record \-\-call-graph=dwarf ./command
perf report

This post intentionally left blank.

We ran out of Yakking articles and energy to write new ones. There's plenty of things to write about, but the usual Yakking writers have been feeling a bit under the weather lately. If you'd like to help, leave a comment on Yakking, or join the IRC channel #yakking (on the irc.oftc.net network), and offer an article.

The previous break in the weekly schedule was December 25, 2013. That was due to a typo in the intended publication date: it got scheduled for 2103-12-25, not 2013-12-25.

We hope to return back to the normal schedule. Your patience is appreciated. Resistance less than 4.2 Ohm is futile.

You've started a new project. When should you start using it "in anger", "for real", "in production"? My advice is to do so as early as you can.

I did this for my newest project. I've been developing it slowly, and things had matured enough that it could now actually do things. I set up two instances doing things that I, and my company, now rely on. If the software breaks, I will need to take action (disable the broken instances, do things in another way until I fix the software).

The point of doing this early is that it gives me quick feedback on whether the software works at all, and makes it easy to add new features, and makes it easier for others to try out the software. (When I announce it publically, which I'm not yet doing.)

• I see quickly if something doesn't work. (For now, it does.)
• I see at once if there's a missing feature that I urgently need. (I have found two bugs yesterday.)
• I see what is awkward and cumbersome for configuring, deploying the software. (So many interacting components.)
• I see if it's nice to actually use. (I need to find someone to write a web interface, and need to improve the command line tool.)
• I see if the performance is adequate and get an idea what the actual resource requirements are. (Not many resources needed for now. Even the cheapest VM I could choose is adequate.)
• I get more confident in the software the more I actually use it. Writing a test suite is good, but real use is better. Real use always comes up with things you didn't think about writing tests for.

In order to set up not just one but two instances, I had to make the deployment automated. (I'm that lazy, and I don't apologise for that.)

Thanks to an automated setup, when I add features or fix bugs, they're easy to roll out. It's now almost as easy as just running the program from the source tree.

My development process is now, I write tests; I write code; tests pass; I tag a release; I let CI build it; and I run Ansible to upgrade everywhere. About 15 seconds of work once the tests pass, though it takes a couple of minutes of wall-clock time, since I run CI on my laptop.

Apart from the benefits that come from the features of the software itself, getting to this stage is emotionally very rewarding. In one day, my little pet project went from "I like this idea" to "it's a thing".

I recommend you start using your stuff in production earlier rather than later. Do it now, do it every day.

Previously I mentioned the Advent of Code as a possible thing you might want to look at for using as a way to learn a new language in a fun and exciting way during December.

This year, it'll be running again, and I intend to have a go at it again in Rust because I feel like I ought to continue my journey into that language.

It's important to note, though, that I find it takes me between 30 and 90 minutes per day to engage properly with the problems, and frankly the 30 minute days are far more rare than the 90 minute ones. As such, I urge you to not worry if you cannot allocate the time to take part every day. Ditto if you start and then find you cannot continue, do not feel ashamed. Very few Yakking readers are as lucky as I am in having enough time to myself to take part.

However, if you can give up the time, and you do fancy it, then join in and if you want you can join my private leaderboard and not worry so much about competing with the super-fast super-clever people out there who are awake at midnight Eastern time (when the problems are published). If you want to join the leaderboard (which contains some Debian people, some Codethink people, and hopefully by now, some Yakking people) then you will need (after joining the AoC site) to go to the private leaderboard section and enter the code: 69076-d4b54074. If you're really enthusiastic, and lucky enough to be able to afford it, then support AoC via their AoC++ page with a few dollars too.

Regardless of whether you join in with AoC or not, please remember to always take as much pleasure as you can in your coding opportunities, however they may present themselves.