FOSS projects are mostly developed on a volunteer basis.

This makes the currencies by which they are developed: free time and motivation.

Often times you have the free time, but not the motivation. Often this is not from feeling that the work isn't worth doing, but that you feel inadequate to do it.

Don't be disheartened. There's plenty you can do that helps.

  1. Just be there, whether in-person or online.

    You can do whatever else you want while being there, but it's encouraging to not be along in your endeavours.

    You may even find some motivation of your own.

  2. When others are talking about what they want to achieve, respond enthusiastically.

    It makes them more likely to follow-through and do so, and in the very least makes them feel good.

    This does risk making them feel worse if they never get around to it, but sometimes that's sufficient to shame them into action later, and other times it's sufficient to say "these things happen".

  3. Engage in discussion about what others want to achieve.

    It's extremely valuable for refining ideas, so they can implement what they want to do better, it keeps it fresh in their mind so motivation lasts longer, and it leaves a clearer idea of what to do so it may be completed before motivation runs out.

  4. Mention what other people are doing to people who might be interested.

    You could end up with anecdotes of other people thinking it's a cool idea, which when relayed to people doing the work provides their own motivation.

  5. Remind people of the successes they've had.

    It makes people feel good about what they've already done, and can put any issues they are currently struggling with into perspective.

    Lars pointed out that Yakking has published more than 180 articles at a rate of one per week! We've managed to get this far, we can continue for a good while yet.

Posted Wed Mar 1 12:00:07 2017 Tags:

I have the dubious honour of being one of the people, at my place of work, charged with interviewing technical applicants. Without giving the game away too much, I thought I might give a few hints for things I look for in a CV, and in the wider world, when considering and interviewing a candidate.

First a little context - I tend to interview candidates who are applying for higher-level technical roles in the company, and I have a particular focus on those who claim on their CV to have a lot of experience. I start by reading the cover letter and CV looking for hints of F/LOSS projects the applicant has worked with; either as a user or a developer. I like it when an applicant provides a bitbucket, github or gitlab URL for their personal work if they have any; but I really like it when they provide a URL for their own Git server (as you might imagine).

Once I have identified places on the Internet where I might find someone, I look to dig out their internet ghosts and find out what they are up to in the wider F/LOSS world. The best candidates show up in plenty of places, are easily found making nice commits which show their capability, and seem well spoken on mailing lists, fora, et al. Of course, if someone doesn't show up on Internet searches then that doesn't count against them because to have the privilege of being able to work on F/LOSS is not something afforded to all; but if you do show up and you look awful it will count against you.

Also remember, there's more ways to contribute than writing code. I love it when I find candidates have made positive contributions to projects outside of just coding for them. Help a project's documentation, or be part of mentoring or guide groups, and I'll likely be very pleased to talk with you.

Beyond the Internet Stalking, I like to get my candidates to demonstrate an ability to compare and contrast technologies; so a good way to get on my good side is to mention two similar but conflicting capabilities (such as Subversion and Git) be prepared to express a preference between them, and be able to defend that preference.

Finally a few basic tips -- don't lie, dissemble, or over-inflate in your CV or cover-letter (I will likely find out) and don't let your cover letter be more than a single side of A4, nor your CV more than 2 sides of A4.

If I ever interview you, and I find out you read this article, I will be most pleased indeed. (Assuming you take on my recommendations at least :-) )

Posted Wed Mar 8 12:00:07 2017

Bit rot, specifically the phenomenon of software working less well even if it hasn't changed, is annoying, but a fact of life. There might be a thing or two you can do to make it happen less.

Examples from your author's personal experience from the past year:

  • Cloud provider changes the default username on base images from ec2-user to debian, requiring simple changes needed in many places.
  • Cloud provider upgrades their virtualisation platform, which introduces a new API version, and breaks the old version. All API using automation needs upgrading.
  • Configuration management software introduces a new feature (become), and deprecates the old corresponding feature (sudo). Simple changes, but in many places.
  • Configuration management software breaks the new feature (can no longer switch to an unprivileged user to run shell script snippets), requiring more complicated changes in several places (run shell as root, invoke sudo explicitly).
  • Author's software depends on enterprise-grade software for a specific service, which switches to requiring Oracle Java, instead of OpenJDK. Author's software isn't fully free software anymore.

Bit rot happens for various reasons. The most common reason is that the environment changes. For example, software that communicates over the network may cease to function satisfactorily if the other computers change. A common example is the web browser: even though your computer works just as well as before, in isolation, web sites use new features of HTML, CSS, and JavaScript, not to mention media formats, and web pages become bigger, and in general everything becomes heavier. Also, as your browser version ages, sites stop caring about testing with it, and start doing things that expose bugs in your version. Your web experience becomes worse every year. Your browser bit rots.

There is no way to prevent bit rot. It is a constant that everything is variable. However, you can reduce it by avoiding common pitfalls. For example, avoid dependencies that are likely to change, particularly in ways that will break your software. An HTML parsing library will necessarily change, but that shouldn't break your software if the library provdes a stable API. If the library adds support for a new syntactic construction in HTML, your program should continue to work as before.

You should be as explicit as possible in what you expect from the environment. Aim to use standard protocols and interfaces. Use standard POSIX system calls, when possible, instead of experimental Linux-specific ones from out-of-tree development branches. Sometimes that isn't possible: document that clearly.

Have automated ways of testing that your software works, preferably tests that can be run against an installed instance. Run those tests from time to time. This will let you and your users notice earlier that something's broken.

Posted Wed Mar 15 12:00:07 2017 Tags:

Sometimes it is necessary to leave a process running, performing some service in the background while doing something else.

It would be redundant and possibly harmful to start a new one if it is already running.

Ideally all programs would safely shut themselves down if already running, checking if it's running before starting only guarantees that it was runing when you checked, rather than that it is running when you need it. For most purposes though it is reasonable to check first.

So how do we know if our service is running?

You may have run ps(1) before to see if a process is running, so you might naturally think this would be how to do it.

This would of course fall into the trap of parsing the output of shell commands. Why should we write fragile code when ps(1) is using a proper API to do it?

The way this is accomplished is the procfs virtual file system traditionally mounted at /proc. There is a subdirectory in this file system for each process listed by its process ID.

We can list all directories that are processes by running:

find /proc -mindepth 1 -maxdepth 1 -name '[0-9]*'

Inside each of these directories are files describing the process.

Check comm

When you look at the output of ps it shows the name of the process, which is normally the base name of the file path of the executable that the process was started with.

This is stored in the file in /proc called comm.

So if the name of your program is "myprogram", you can find out if your program is running with the following command:

find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
    '[ "$(cat "$1/comm")" = myprogram ] && echo Is running' - {} ';'

I would recommend against checking if your program is running this way though, as processes may call themselves whatever they want, by writing the new name to comm.

$ cat /proc/$$/comm
bash
$ printf dash >/proc/$$/comm
$ cat /proc/$$/comm
dash

This is often used by services that fork off helper processes to name the subprocesses after their role to make it easier for developers or sysadmins to know what they do.

Check exe

The procfs entry also includes the path of the executable the proccess was started from as a symbolic link.

Thus if your program is installed at /usr/bin/myprogram then we can check whether it is running with:

find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
    '[ "$(readink "$1/exe")" = /usr/bin/myprogram ] && echo Is running' - {} ';'

This cannot be modified by the proces after it has started, but as usual caveats apply:

  1. Not all processes have an initial executable. This symbolic link may be unreadable (fails with errno of ENOENT) in the case of kernel threads.

  2. It could be a program that has subcommands, one of which may be a long-running service (e.g. git-daemon), which you wouldn't want to fail to start just because a shorter operation with a different subcommand happened to be running at the same time.

  3. This is unhelpful in the case of interpreted languages, since it is always the name of the interpreter rather than the name of the script.

  4. The same program may be reachable by multiple file paths if the executable has been hard-linked.

  5. If the program's executable may be removed while it is running, changing exe to append " (deleted)" to the file path.

    If this file is then replaced then another process may have the same executable path but an incompatible behaviour.

    This isn't even unusual if the name of the process is generic, like "sh" or "httpd".

So it's useless for interpreted programs and unreliable if the executable can be replaced.

Check cmdline

It could be perfectly safe to run the same program multiple times provided it is passed different configuration.

The cmdline file can be parsed to infer this configuration as a list of strings that are NUL terminated.

A problem with this approach is the need to reimplement parsing logic and know for all command-lines whether it's appropriate to start another.

This logic could be quite difficult, but you could add a parameter just for determining whether it is the same.

This is far from ideal because:

  1. Lookup time gets worse as your system has more processes running.
  2. Processes can modify their command-line too, so another process could arrange to have the same command-line, and make this unreliable.

Next time we are going to look at a better use for that parameter.

Posted Wed Mar 22 12:00:08 2017 Tags: