Daniel Silverstone Four years of chinwaggery

Four years ago, Lars posted our first article on software freedom. This started an epic effort to help educate those who want to get into free-software. We pledged to discuss technical topics such as shell and C, and non-technical topics such as diversity and motivation. Through our time we've had opinionated series on the truisms of software, or strong technical sets around topics such as systemd.

To give you some numbers, in the past four years we've published two hundred and eleven (or two hundred and twelve if you include this one) articles, by seven authors. Eighty-three were by Richard Maw, seventy-four (seventy-five) by yours truly, and forty-one by Lars Wirzenius. Our most prolific guest author was Will Holland who has contributed seven, Richard Ipsum and Jonathan Maw have contributed two each. Jon Dowland has contributed one, and one of our articles was deliberately anonymously posted one Christmas.

We've received thirty-eight comments spread over twenty-four of the articles. Our most prolific commenter has been Richard Ipsum, and otherwise they're spread relatively thinly among the approximately nineteen other commenters. A total of fifteen of the thirty-eight comments were by authors, usually in response to comments on their articles.

We have six items in our suggestion-box (thank you) though some are us acknowledging comments. We're grateful for every suggestion and every bit of encouragement you've provided over the years, via the site, IRC, Twitter, etc.

Over the years, our core contributors have had their own ups and downs, and yet we've pretty much managed to get an article out every week no matter what. Indeed we missed one week (I believe) in all that time, and that was because we typoed the date in our publishing queue, and none of us caught it in review. Sadly, it gets harder and harder though, and so please do pop over to the suggestion box if there're topics you'd like to see us cover (even if you just want an expansion of something we've previously discussed). Perhaps come and join us in #yakking on the OFTC IRC network, and if you're feeling very enthusiastic then come and discuss guest-posting since we're always grateful for contributions.

Here's to another four years of articles which you consider worth reading…

Posted Tue Aug 22 12:00:08 2017
Daniel Silverstone Taking time for yourself

Occupational Burnout is a thing which people worry about a lot in the "real world" and it's why we encourage people, particularly information workers, to take regular breaks, to not think about their job outside of work. Indeed, burnout has been referred to as a disease and as such we recommend strongly that you avoid it.

Sadly, those of us who are blessed with the ability and the interest in contributing to F/LOSS tend to work hard, and then effectively "work as play" hard too; which can lead to burning out even more quickly. It can be even worse when the stress originates in your F/LOSS activity because it can be seen as making your 'funtime' less enjoyable.

Burning out is a leading cause of people having to retire from F/LOSS projects and as such, it's really important that you know when to step away from F/LOSS and have a break. You might prefer to spend a few days entirely away from your computer or just spend it immersed in a game. If you're particularly lucky/unlucky then you might just be able to work on a different project for a little while instead. Whatever you do to relax, be sure to increase your explicit relaxation time along with time you're spending on F/LOSS to keep yourself happy and healthy, and able to contribute for a nice long time to come.

Your homework this week is to just think about what you enjoy doing which isn't work or F/LOSS related at all, and go do it. Just shoo!

Posted Wed Aug 16 12:00:06 2017
Lars Wirzenius Retiring from a project

Sometimes you'll want to leave a free software project. It might be one you founded or one you've joined. You may have spent years contributing to it. You may have formed friendships with other contributors: working together on something for a long time tends to be a catalyst for that. However, things change and you may want to leave. You might no longer be using the software. You might not have time to contribute. There might be disagreements about where the project is going or how it is going to be operating. You might be moving to a different continent. You may be switching careers entirely. You may be founding an unrelated company.

A term for this is 'retiring'. Depending on the project, there may be a processs to follow, or you may just wing it.

More importantly, there're various aspects to consider when retiring, especially if you've been involved for a long time.

  • Why are you leaving? It's best to be honest about this, particularly to yourself. Don't rage quit.
  • Will the project survive you leaving?
  • How will users of the software be affected?
  • What about other collaborators on the project?
  • Will there be people to pick up the slack and take over responsibilities? Will they know what to do? Can they ask you for help afterwards?
  • Is there any publicity likely to follow from the retirement? (Probably not, except for high-profile projects.)
  • Are there any assets (computers, etc) that need to be dealt with?

Retiring from a free software project is a lot like leaving paid employment. If you do it well, you make sure all your old commitments and responsibilities are handed over to new people, and no-one is affected too adversely from the change. As a result, you'll be remembered with fondness and you're always welcome back.

Unlike paid employment, there's few hard and fast rules in Free Software. It's important to remember that though your contributions are valued, you're not obliged to continue them if you don't want to.

Posted Wed Aug 9 12:00:08 2017
Daniel Silverstone My F/LOSS Activity...

There has been a growing trend among bloggers, particularly among those aggregated on Planet Debian, in which the blogger writes regularly (usually monthly) about their activity in the Free Software communities since their last report.

This can be used as a way to "keep yourself honest" in the sense that if you're committed to reporting your activity on a month by month basis, you kinda need some activity to report. It can be used as a way to keep your colleagues and co-conspiritors in F/LOSS informed about what you're working on, and cool things you've done which they might be interested in. It might be a useful way to motivate yourself when each period of time you get to write a nice blog posting all about the wonderful things you've achieved and the projects you've improved along the way.

Recently I've been persuaded to join the trend and I'm looking forward to experiencing the effects I've mentioned above. I'd like to suggest that all of you go out and write yourself a blog post all about the wonderful things you've done and then commit yourself to doing so again in X amount of time in the future.

Posted Wed Aug 2 12:00:07 2017
Richard Maw Time - Adjustment

CLOCK_MONOTONIC_RAW reflects the underlying hardware for measuring time.

In the short term this is mostly correct, but hardware isn't perfect, so it can, over long periods, drift and no longer be synchronised with what the rest of the world considers the time to be.

If you know what the time is meant to be elsewhere in the world, then you can adjust your clock to correct it.

Typically your computer will do this with the Network Time Protocol, or NTP,, by asking trusted computers on the internet what the time is.

Actually correcting the time works by using the adjtimex(2) or clock_adjtime(2) system calls.

This can't be done with CLOCK_MONOTONIC_RAW, but can be done with CLOCK_MONOTONIC.

In addition to being correctable, CLOCK_MONOTONIC can be used with clock_nanosleep(2).

This will allow you to sleep for at least the period of time specified, though could be interrupted when a signal is delivered, (which could happen with timer_create(2)).

Similar clocks

If it's more important to get the time quickly, than to get a more precise time, such as if you're profiling real-time software and want to not slow it down, then you can use CLOCK_MONOTONIC_COARSE.

CLOCK_MONOTONIC can be used to time events, but works by counting seconds while the computer is running. This could be a problem if your computer suspends, since then it would stop counting.

The solution to this is the CLOCK_BOOTTIME clock, which will include seconds spent suspended, so could be used to time external events.

So far everything discussed has been somewhat abstract, divorced from what we commonly understand to be time.

This will be rectified in the next article in the time series, where we will be covering "real time".

Posted Wed Jul 26 12:00:07 2017 Tags:

Normally our Unix systems organise the file system in a structure called the Filesystem Hierarchy Standard (FHS). Installing into an FHS has limitations, what would happen if we want to install, for example, two different versions of ruby at the same time? Typically this isn't possible without explicitly specifying a separate installation directory, if we just install to the usual place e.g. /usr/bin then we will just overwrite the previous ruby. So perhaps we would install one ruby into /usr/bin and another into /usr/local/bin, this is fine, but what about dependent libs? Assuming the two different versions of ruby do require different dependencies then we have potentially the same problem that the dependencies for the 1st ruby might overwrite the dependencies for the 2nd ruby.

Nix gets around this to some extent by not using FHS, instead nix installs all files into the nix store, which is usually located at /nix/store. All programs in a nix store are identified by their store path, which is uniquely generated for each distinct nix package. As a result of this, different versions of the same ruby no longer conflict because they are each assigned their own locations within the nix store.

To enable use of programs within the store, nix maintains an environment which is basically a mapping of FHS path -> nix store path, where the -> is a symlink. So for example, let's first install ruby 2.0 into our environment

nix@salo:~$ nix-env -f nixpkgs -iA pkgs.ruby_2_0
installing ‘ruby-2.0.0-p648’
these paths will be fetched (3.43 MiB download, 19.35 MiB unpacked):
fetching path ‘/nix/store/bxm4s71qdyh071ap5ywxc63aja62cbyc-gdbm-1.13’...
building path(s) ‘/nix/store/j649f78ha04mi1vykz601b00ml3qlr9q-user-environment’
created 419 symlinks in user environment

we can see the symlink that was just created to our ruby2.0 in the store,

nix@salo:~$ ls -l $(which irb)
lrwxrwxrwx 1 nix nix 67 Jan  1  1970 /home/nix/.nix-profile/bin/irb -> /nix/store/d2ccapssrq683rj0fr7d7nb3ichxvlsy-ruby-2.0.0-p648/bin/irb

nix@salo:~$ irb
irb(main):001:0> puts RUBY_VERSION

as you can see we're only able to execute the interactive ruby prompt irb because it's symlinked into our environment which is, of course, on the $PATH,

nix@salo:~$ echo $PATH

to prove the point about multiple versions on the same system let's swap ruby 2.0 for ruby 2.4

nix@salo:~$ nix-env -f nixpkgs -iA pkgs.ruby_2_4
replacing old ‘ruby-2.0.0-p648’
installing ‘ruby-2.4.1’
these paths will be fetched (3.13 MiB download, 15.32 MiB unpacked):
fetching path ‘/nix/store/48xrfkanmx5sshqj1364k2dw25xr4znj-ruby-2.4.1’...
*** Downloading ‘https://cache.nixos.org/nar/00hh9w9nvlbinya1i9j0v7v89pw3zzlrfqps72441k7p2n8zq7d3.nar.    xz’ to ‘/nix/store/48xrfkanmx5sshqj1364k2dw25xr4znj-ruby-2.4.1’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 3205k  100 3205k    0     0   114k      0  0:00:27  0:00:27 --:--:--  125k

building path(s) ‘/nix/store/7b2mmk2ffmy1c2bxq7r6y9cn6r0nwn8s-user-environment’
created 415 symlinks in user environment

nix@salo:~$ ls -l $(which irb)
lrwxrwxrwx 1 nix nix 62 Jan  1  1970 /home/nix/.nix-profile/bin/irb -> /nix/store/48xrfkanmx5sshqj1364k2dw25xr4znj-ruby-2.4.1/bin/irb

nix@salo:~$ irb
irb(main):001:0> puts RUBY_VERSION

You may be wondering whether this is really an improvement, since although we have multiple versions of the same package installed on our system, we can only have one ruby in the environment at any one time. To deal with this nix provides the nix-shell utility which constructs an environment on demand and runs a new shell based on that environment.

nix@salo:~/nixpkgs$ nix-shell -p ruby_2_0            
these paths will be fetched (4.44 MiB download, 22.57 MiB unpacked):
fetching path ‘/nix/store/m74m7c4qbzml7ipfxzlpxddcn9ah8jrs-gdbm-1.12’...
[nix-shell:~/nixpkgs]$ which irb

[nix-shell:~/nixpkgs]$ irb
irb(main):001:0> puts RUBY_VERSION
=> nil

nix@salo:~/nixpkgs$ nix-shell -p ruby_2_4
these paths will be fetched (3.13 MiB download, 15.30 MiB unpacked):
fetching path ‘/nix/store/wly748apb5r37byvvgq85hshgzcahv0y-ruby-2.4.0’...
[nix-shell:~/nixpkgs]$ which irb

[nix-shell:~/nixpkgs]$ irb
irb(main):001:0> puts RUBY_VERSION
=> nil

We haven't even started to scratch the surface in this intro, there's lots of really exciting stuff I've not even mentioned, like how you can always rollback to the environment at an earlier state: every mutation to the environment is recorded, so every time you install or uninstall a nixpkg a new "generation" of the environment is created, and it's always possible to immediately rollback to some earlier generation. NixOS itself takes all these super exciting ideas and applies them to an entire operating system, where each user has their own environment, so ruby for one user might mean ruby2.0 and ruby for another might mean ruby2.4. Hopefully it's clear now how these different versions of the same package can live in harmony under NixOS.

I hope I've managed to convey some of nix's coolness in this short space, if I have then you should definitely check lethalman's "nix-pills"1 series for a really deep explanation of how nix works internally and how to create nixpkgs from scratch. There's also ofcourse the NixOS website2 and #nixos on irc.freenode.net which is probably one of the friendliest communities out there.

Posted Wed Jul 19 12:00:07 2017

System calls

You can't talk about time without clocks. A standard definition of clocks is "An instrument to measure time".

Strictly speaking the analogy isn't perfect, since these clocks aren't just different ways of measuring time, they measure some different notions of time, but this is a somewhat philosophical point.

The interface for this in Linux are system calls with a clkid_t parameter, though not all system calls are valid for every clock.

This is not an exhaustive list of system calls, since there are also older system calls for compatibility with POSIX or other UNIX standards, that offer a subset of the functionality of the above ones.

System calls beginning clock_ are about getting the time or waiting, beginning timer_ are for setting periodic or one-shot timers, timerfd_ are for timers that can be put in event loops.

For the sake of not introducing too many concepts at once, we're going to start with the simplest clock and work our way up.


The first clock we care about is CLOCK_MONOTONIC_RAW.

This is the simplest clock.

It it initialised to an arbitrary value on boot and counts up at a rate of one second per second to the best of its ability.

This clock is of limited use on its own since the only clock-related system calls that work with it are clock_gettime(2) and clock_getres(2). It can be used to determine the order of events, if the time of the event was recorded and the relative time difference between when they happened, since we know that the clock increments one second per second.


In this program below, we time how long it takes to read 4096 bytes from /dev/zero, and print the result.

#include <stdio.h> /* fopen, fread, printf */
#include <time.h> /* clock_gettime, CLOCK_MONOTONIC_RAW, struct timespec */

int main(void) {
    FILE *devzero;
    int ret;
    struct timespec start, end;
    char buf[4096];

    devzero = fopen("/dev/zero", "r");
    if (!devzero) {
        return 1;

    /* get start time */
    ret = clock_gettime(CLOCK_MONOTONIC_RAW, &start);
    if (ret < 0) {
        return 2;

    if (fread(buf, sizeof *buf, sizeof buf, devzero) != sizeof buf) {
        return 3;

    /* get end time */
    ret = clock_gettime(CLOCK_MONOTONIC_RAW, &end);
    if (ret < 0) {
        return 4;

    end.tv_nsec -= start.tv_nsec;
    if (end.tv_nsec < 0) {
        end.tv_nsec += 1000000000l;
    end.tv_sec -= start.tv_sec;

    printf("Reading %zu bytes took %.0f.%09ld seconds\n",
           sizeof buf, (double)end.tv_sec, (long)end.tv_nsec);

    return 0;

You're possibly curious about the naming of the clock called MONOTONIC_RAW.

In the next article we will talk about CLOCK_MONOTONIC, which may help you understand why it's named the way it is.

I suggested the uses of this clock are for the sequencing of events and for calculating the relative period between them. If you can think of another use please send us a comment.

If you like to get hands-on, you may want to try reimplementing the above program in your preferred programming language, or extending it to time arbitrary events.

Posted Wed Jul 12 12:00:13 2017 Tags:

Whether I am writing my own program, or chosing between existing solutions, one aspect of the decision making process which always weighs heavily on my mind is that of the input and output data formats.

I have been spending a lot of my work days recently working on converting data from a proprietary tool's export format into another tool's input format. This has involved a lot of XML diving, a lot more swearing, and a non-trivial amount of pain. This drove home to me once more that the format of input and output of data is such a critical part of software tooling that it must weigh as heavily as, or perhaps even more heavily than, the software's functionality.

As Tanenbaum tells us, the great thing about standards is that there's so many of them to choose from. XKCD tells us, how that comes about. Data formats are many and varied, and suffer from specifications as vague as "plain text" to things as complex as the structure of data stored in custom database formats.

If you find yourself writing software which requires a brand new data format then, while I might caution you to examine carefully if it really does need a new format, you should ensure that you document the format carefully and precisely. Ideally give your format specification to a third party and get them to implement a reader and writer for your format, so that they can check that you've not missed anything. Tests and normative implementations can help prop up such an endeavour admirably.

Be sceptical of data formats which have "implementation specific" areas, or "vendor specific extension" space because this is where everyone will put the most important and useful data. Do not put such beasts into your format design. If you worry that you've made your design too limiting, deal with that after you have implemented your use-cases for the data format. Don't be afraid to version the format and extend later; but always ensure that a given version of the data format is well understood; and document what it means to be presented with data in a format version you do not normally process.


Given all that, I exhort you to consider carefully how your projects manage their input and output data, and for these things to be uppermost when you are choosing between different solutions to a problem at hand. Your homework is, as you may have grown to anticipate at this time, to look at your existing projects and check that their input and output data formats are well documented if appropriate.

Posted Wed Jul 5 12:00:06 2017 Tags:
Richard Maw What is Time?

Time is a big enough subject that I'd never finish writing about it.

Fortunately this is a Linux technical blog, so I only have to talk about it from that perspective.

In a departure from our usual article series format, we are going to start with an introductory article and follow up starting from basics, introducing new concepts in order of complexity.

What is time?

Time is seconds since the unix epoch.

The Unix epoch is midnight, Thursday, 1st January 1970 in the UTC time zone.

If you know the unix time and how many seconds in each day since then you can use the unix timestamp to work out what time it is now in UTC time.

If you want to know the time in another timezone the naïve thing is to apply a timezone offset based on how far or behind the zone is compared to UTC.

This is non-trivial since time is a political issue, requiring global coordination with hundreds of countries coming in and out of [daylight saving time][] and various other administrative changes, time often changes.

Some times don't exist in some timezones. Samoa skipped a day and changed time zone.

A side-effect of your system clock being in Unix time is that to know local time you need to know where you are.

The talk linked to in https://fosdem.org/2017/schedule/event/python_datetime/ is primarily about time in python, but covers the relationship between time stamps, clock time and time zones.

http://www.creativedeletion.com/2015/01/28/falsehoods-programmers-date-time-zones.html is a good summary of some of the common misconceptions of time zone handling.

Now that we've covered some of the concepts we can discuss in future articles how this is implemented in Linux.

Posted Wed Jun 28 12:00:07 2017 Tags:

A lot of people keep a diary. This is a time-tested way of ensuring future historians have juicy rumours to read and write books about. A journal is like a diary, but it's meant to be more serious and less saucy.

I've been keeping a journal for some years now. It's been good in several ways:

  • It's a private place where I can vent entirely without any inhibition. As long as I don't leak the content, I can write anything there, things I couldn't share even with my closest friends and loved ones. In my journal I don't need to be fair or balanced or diplomatic; if calling my boss a poophead helps me, I can do that in my journal. More importantly, apart from name-calling, I can be open in my journal about my hopes and dreams, and speculate as freely as I need to about all sorts of crazy ideas. If I want to fantasize about writing my own Debian installer, my journal is where I'll do it. In my journal I don't have to worry about people misunderstanding me, or attacking any vague, half-developed crazy ideas, and ridicule me about them for the next several years.

    My journal is a safe place. (This is one of the reasons why my backups are encrypted.)

  • It's a place to keep an external memory. One of the the things I put in my journal is a stream of consciousness while developing. This allows me to answer questions of the form "what the bleeding heck was I thinking then designing this software" with quotes from my younger self. Perhaps more usefully, this can be applied to debugging as well: tricky problems often involve a lot of data to be kept to fully understand what's going on, and a written journal is a better place for that than the brain.

    A bug tracker is usually not a good place for this, or not the full stream of consciousness. Most of that stream is necessary for the process, but other people shouldn't be flooded with all of it, only the actually relevant parts.

  • I also put all sorts of other bits of information into my journal. In fact, over the years it has developed into a personal knowledge base, where I can find a bunch of things that are relevant to me, but not necessarily easy to find online. As an example, what's the command to do TOTP on Linux that works with Amazon? (oathtool --base32 --totp, not too difficult to find but easier in my journal.)

    Some information is private in nature, such as who were the people I had that interesting lunch with at that conference that one time.

There are a myriad of tools for keeping a journal, or personal knowledge base. There's wikis of various flavors, plain text files in git, online document services, etc etc. You can keep it on paper as well, though that makes grepping harder. As I prefer to not pour my heart and soul into a service run by other people, my journal is an ikiwiki instance that I run on my laptop, which renders a static HTML that is served via Apache on my laptop (and only to my laptop). This satisfies my needs for ease of use and privacy.

Using a wiki engine for this is nice, because linking adds a lot of power and can make finding relevant information faster. Using the ikiwiki inline directive, which produces pages by collecting other pages based on a pattern, I get pages for particular people (link to the person, the person's page includes the page linking to them), topics (e.g., projects), tags, and more. Quite nifty, and I'm afraid I can't show you.

Keeping a journal takes a bit of effort, of course. It also takes time for a journal to become useful: having diary entries for a week probably doesn't help much. Having them from a decade changes this in a qualitative, not just a quantitative way. Do you remember what you got your loved as a present seven years ago? I don't even remember what I got as a present last year.

Give it a try. It doesn't have to be perfect, but you need to keep doing it. Report back in a comment below in four years from now and tell use if it was helpful to you.

Posted Wed Jun 21 12:00:08 2017 Tags: