FOSS projects are mostly developed on a volunteer basis.

This makes the currencies by which they are developed: free time and motivation.

Often times you have the free time, but not the motivation. Often this is not from feeling that the work isn't worth doing, but that you feel inadequate to do it.

Don't be disheartened. There's plenty you can do that helps.

  1. Just be there, whether in-person or online.

    You can do whatever else you want while being there, but it's encouraging to not be along in your endeavours.

    You may even find some motivation of your own.

  2. When others are talking about what they want to achieve, respond enthusiastically.

    It makes them more likely to follow-through and do so, and in the very least makes them feel good.

    This does risk making them feel worse if they never get around to it, but sometimes that's sufficient to shame them into action later, and other times it's sufficient to say "these things happen".

  3. Engage in discussion about what others want to achieve.

    It's extremely valuable for refining ideas, so they can implement what they want to do better, it keeps it fresh in their mind so motivation lasts longer, and it leaves a clearer idea of what to do so it may be completed before motivation runs out.

  4. Mention what other people are doing to people who might be interested.

    You could end up with anecdotes of other people thinking it's a cool idea, which when relayed to people doing the work provides their own motivation.

  5. Remind people of the successes they've had.

    It makes people feel good about what they've already done, and can put any issues they are currently struggling with into perspective.

    Lars pointed out that Yakking has published more than 180 articles at a rate of one per week! We've managed to get this far, we can continue for a good while yet.

Posted Wed Mar 1 12:00:07 2017 Tags:

I have the dubious honour of being one of the people, at my place of work, charged with interviewing technical applicants. Without giving the game away too much, I thought I might give a few hints for things I look for in a CV, and in the wider world, when considering and interviewing a candidate.

First a little context - I tend to interview candidates who are applying for higher-level technical roles in the company, and I have a particular focus on those who claim on their CV to have a lot of experience. I start by reading the cover letter and CV looking for hints of F/LOSS projects the applicant has worked with; either as a user or a developer. I like it when an applicant provides a bitbucket, github or gitlab URL for their personal work if they have any; but I really like it when they provide a URL for their own Git server (as you might imagine).

Once I have identified places on the Internet where I might find someone, I look to dig out their internet ghosts and find out what they are up to in the wider F/LOSS world. The best candidates show up in plenty of places, are easily found making nice commits which show their capability, and seem well spoken on mailing lists, fora, et al. Of course, if someone doesn't show up on Internet searches then that doesn't count against them because to have the privilege of being able to work on F/LOSS is not something afforded to all; but if you do show up and you look awful it will count against you.

Also remember, there's more ways to contribute than writing code. I love it when I find candidates have made positive contributions to projects outside of just coding for them. Help a project's documentation, or be part of mentoring or guide groups, and I'll likely be very pleased to talk with you.

Beyond the Internet Stalking, I like to get my candidates to demonstrate an ability to compare and contrast technologies; so a good way to get on my good side is to mention two similar but conflicting capabilities (such as Subversion and Git) be prepared to express a preference between them, and be able to defend that preference.

Finally a few basic tips -- don't lie, dissemble, or over-inflate in your CV or cover-letter (I will likely find out) and don't let your cover letter be more than a single side of A4, nor your CV more than 2 sides of A4.

If I ever interview you, and I find out you read this article, I will be most pleased indeed. (Assuming you take on my recommendations at least :-) )

Posted Wed Mar 8 12:00:07 2017

Bit rot, specifically the phenomenon of software working less well even if it hasn't changed, is annoying, but a fact of life. There might be a thing or two you can do to make it happen less.

Examples from your author's personal experience from the past year:

  • Cloud provider changes the default username on base images from ec2-user to debian, requiring simple changes needed in many places.
  • Cloud provider upgrades their virtualisation platform, which introduces a new API version, and breaks the old version. All API using automation needs upgrading.
  • Configuration management software introduces a new feature (become), and deprecates the old corresponding feature (sudo). Simple changes, but in many places.
  • Configuration management software breaks the new feature (can no longer switch to an unprivileged user to run shell script snippets), requiring more complicated changes in several places (run shell as root, invoke sudo explicitly).
  • Author's software depends on enterprise-grade software for a specific service, which switches to requiring Oracle Java, instead of OpenJDK. Author's software isn't fully free software anymore.

Bit rot happens for various reasons. The most common reason is that the environment changes. For example, software that communicates over the network may cease to function satisfactorily if the other computers change. A common example is the web browser: even though your computer works just as well as before, in isolation, web sites use new features of HTML, CSS, and JavaScript, not to mention media formats, and web pages become bigger, and in general everything becomes heavier. Also, as your browser version ages, sites stop caring about testing with it, and start doing things that expose bugs in your version. Your web experience becomes worse every year. Your browser bit rots.

There is no way to prevent bit rot. It is a constant that everything is variable. However, you can reduce it by avoiding common pitfalls. For example, avoid dependencies that are likely to change, particularly in ways that will break your software. An HTML parsing library will necessarily change, but that shouldn't break your software if the library provdes a stable API. If the library adds support for a new syntactic construction in HTML, your program should continue to work as before.

You should be as explicit as possible in what you expect from the environment. Aim to use standard protocols and interfaces. Use standard POSIX system calls, when possible, instead of experimental Linux-specific ones from out-of-tree development branches. Sometimes that isn't possible: document that clearly.

Have automated ways of testing that your software works, preferably tests that can be run against an installed instance. Run those tests from time to time. This will let you and your users notice earlier that something's broken.

Posted Wed Mar 15 12:00:07 2017 Tags:

Sometimes it is necessary to leave a process running, performing some service in the background while doing something else.

It would be redundant and possibly harmful to start a new one if it is already running.

Ideally all programs would safely shut themselves down if already running, checking if it's running before starting only guarantees that it was runing when you checked, rather than that it is running when you need it. For most purposes though it is reasonable to check first.

So how do we know if our service is running?

You may have run ps(1) before to see if a process is running, so you might naturally think this would be how to do it.

This would of course fall into the trap of parsing the output of shell commands. Why should we write fragile code when ps(1) is using a proper API to do it?

The way this is accomplished is the procfs virtual file system traditionally mounted at /proc. There is a subdirectory in this file system for each process listed by its process ID.

We can list all directories that are processes by running:

find /proc -mindepth 1 -maxdepth 1 -name '[0-9]*'

Inside each of these directories are files describing the process.

Check comm

When you look at the output of ps it shows the name of the process, which is normally the base name of the file path of the executable that the process was started with.

This is stored in the file in /proc called comm.

So if the name of your program is "myprogram", you can find out if your program is running with the following command:

find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
    '[ "$(cat "$1/comm")" = myprogram ] && echo Is running' - {} ';'

I would recommend against checking if your program is running this way though, as processes may call themselves whatever they want, by writing the new name to comm.

$ cat /proc/$$/comm
bash
$ printf dash >/proc/$$/comm
$ cat /proc/$$/comm
dash

This is often used by services that fork off helper processes to name the subprocesses after their role to make it easier for developers or sysadmins to know what they do.

Check exe

The procfs entry also includes the path of the executable the proccess was started from as a symbolic link.

Thus if your program is installed at /usr/bin/myprogram then we can check whether it is running with:

find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
    '[ "$(readink "$1/exe")" = /usr/bin/myprogram ] && echo Is running' - {} ';'

This cannot be modified by the proces after it has started, but as usual caveats apply:

  1. Not all processes have an initial executable. This symbolic link may be unreadable (fails with errno of ENOENT) in the case of kernel threads.

  2. It could be a program that has subcommands, one of which may be a long-running service (e.g. git-daemon), which you wouldn't want to fail to start just because a shorter operation with a different subcommand happened to be running at the same time.

  3. This is unhelpful in the case of interpreted languages, since it is always the name of the interpreter rather than the name of the script.

  4. The same program may be reachable by multiple file paths if the executable has been hard-linked.

  5. If the program's executable may be removed while it is running, changing exe to append " (deleted)" to the file path.

    If this file is then replaced then another process may have the same executable path but an incompatible behaviour.

    This isn't even unusual if the name of the process is generic, like "sh" or "httpd".

So it's useless for interpreted programs and unreliable if the executable can be replaced.

Check cmdline

It could be perfectly safe to run the same program multiple times provided it is passed different configuration.

The cmdline file can be parsed to infer this configuration as a list of strings that are NUL terminated.

A problem with this approach is the need to reimplement parsing logic and know for all command-lines whether it's appropriate to start another.

This logic could be quite difficult, but you could add a parameter just for determining whether it is the same.

This is far from ideal because:

  1. Lookup time gets worse as your system has more processes running.
  2. Processes can modify their command-line too, so another process could arrange to have the same command-line, and make this unreliable.

Next time we are going to look at a better use for that parameter.

Posted Wed Mar 22 12:00:08 2017 Tags:

From time to time, you write something which you really want to show off. If it's a GUI application then you might start with some good screenshots, or possibly even make a screen-capture video of you using the application, so that people can watch and marvel at your amazing creation.

If your wonderful program happens to be a command-line tool, then such videos are often a little lackluster or simply overly-large for the information they convey. It's better if the text being displayed is actually text and that it can be copy-pasted out (so others can follow along in their own terminals). To that end, there is a number of tools for recording operations in a terminal and allowing that to played back. I will mention a few of them here, but this is by no means an exhaustive list; and I shall endeavour to not recommend any specific option over any other.

Asciinema

Asciinema is a pretty slick project housed at https://asciinema.org/ with its source available at https://github.com/asciinema/asciinema and https://github.com/asciinema/asciinema-player. Asciinema is available in Debian and other Linux distributions and works very well. The Asciinema community provides a hosted system and also the tooling necessary to provision the playback of recordings on your own website providing you can honour the GPLv3. The client application is written in Python and has few dependencies.

You can see an example at: https://asciinema.org/a/42383

showterm

Like Asciinema, showterm provides a hosted service, over at https://showterm.io/ and an open-source tool hosted at https://github.com/ConradIrwin/showterm. Showterm's server-side is also open, but is focussed on a hosted experience so expect to be running up a rails app if you want to host it yourself.

Showterm is written in Ruby and is under the MIT licence as is its server. It also relies on ttyrec in some circumstances.

You can see an example at: https://showterm.io/7b5f8d42ba021511e627e

TermRecord

If you're hoping for standalone HTML output, then TermRecord might be what you're after. It can be found at https://github.com/theonewolf/TermRecord and is written in Python. Unlike the other offerings, TermRecord produces entirely standalone output which doesn't depend on any other JavaScript or server resources.

The licence terms for the TermRecord output include MIT licensed JavaScript, and font content under the Ubuntu Font License 1.0.

You can see an example at: http://theonewolf.github.io/TermRecord/figlet-static.html

TTYGIF

The TTYGIF project deserves an honourable mention because despite its output not being copy/pasteable; an animated GIF is much smaller than a video file would likely be. TTYGIF is housed at https://github.com/icholy/ttygif and produces a simple animated GIF of the terminal it is running in. Since almost every web browser which isn't purely text-based offers animated GIF playback, the output of TTYGIF doesn't need any additional support. No JavaScript players or complex plugins.

TTYGIF is MIT licensed, but doesn't seem to place any particular licence on its output. It's also written in C and needs a few dependencies to get itself going, one of which is a tty recorder anyway :-)

You can see an example at: https://camo.githubusercontent.com/acff4cc740350a784cb4539e501fcce1815329c0/687474703a2f2f692e696d6775722e636f6d2f6e76454854676e2e676966

Others

There are plenty of other options, including those which tend to only offer playback in a terminal themselves, such as script and ttyrec. Your homework (You didn't think you'd got away with another week without homework did you?) is to have a play with some of the options listed here (or any others) you can find, and then perhaps comment on this posting showing what you've managed to get up to while creating a tty-cast to demonstrate your latest impressive terminal skills.

Posted Wed Mar 29 11:00:06 2017 Tags: