Daniel Silverstone Writing for others

We've often spoken about how it's good to write software which others will want to use. This is really important but it's only one of the various undertakings which will transform you from a hacker into a good software author. What your users want the software to do is only one aspect of their needs when it comes to any programming you do. In addition there's the aspect of where they want it to work. Now, I'm not going to suggest that you must make your software work everywhere. But I am going to suggest that you need to consider where your users live in terms of their software environment.

If you're having to use lots of bleeding edge libraries in your software development then your users will need to have those libraries in order to use your software. If you're targetting people who use Arch or similar then that might be just fine, but if you're hoping that people who use the stable version of Debian or Fedora then you might need to consider using less bleeding-edge dependencies.

Another solution could be to include the libraries you depend on inside your software, but that tends to make it significantly harder to manage your software as part of a distribution. Or you might make your build system download the new libraries as part of compiling/installing your software in the hope that will help. Unfortunately many build systems are deliberately not allowed to talk to the Internet to improve reproducibility which will scupper your plans.

Some languages have their own packaging solution which can be used to automatically deal with this to a greater or lesser extent, but they have the side-effect of constructing confusing messes on your users' systems unless they are proficient with ways to construct virtual environments for whatever language you have selected.

Instead, the best option for increasing the usability of your software is, simply, to ensure that you don't depend on things which are not already in the distributions you are targetting. This doesn't mean that you must limit yourself to the stable version of a distribution; particularly if you're targetting your software at the distribution's next release; but if you can limit yourself thusly then it will be must easier to have a wide user base. This option also has the benefit that if there's a security issue in any of your dependencies, the distribution will take care of that for you, reducing one potential source of headaches for you.

Posted Wed Jan 4 12:00:08 2017

Unix systems have a number of traditions and conventions for what goes where in the filesystem. Linux follows Unix.

A Unix system has directories such as /bin and /usr/bin for executables, and the difference may be a bit mysterious. It gets worse, though, once you realise there is at least six other directories for executables: /sbin, /usr/sbin, /usr/local/bin, /usr/local/sbin.

In addition to its binaries, a program may consist of data files, shared libraries, and documention, and it may need to write various files for different purposes. The Unix tradition is to spread these into different areas of the filesystem tree.

Some other operating systems have a different approach, often one where all the parts of a program are put in the same place. Unix instead combines like with like: executables in one place (or six), documentation in another, and data files in a third. A common justification is that, say, backing up only the data files becomes easier: everything else can just be reinstalled, if need be.

Early on in the history of Linux, there were big differences between where different Linux distributions put files. A consensus grew that a standard would be beneficial, and so FSSTND, the Linux file system standard, was born. Some years later, its scope was broadened to all free software implementations of Unix and it got renamed to FHS.

The FHS Wikipedia page gives an overview. The actual standard is quite readable and is recommended for when you have time. Additionally the hier(7) also gives an overview. We refer you to those rather than writing our own summary

Things in the filesystem standards world change. The directory tree is alive. For example:

  • /var/run (runtime information, cleared on boot) is moving to /run since keeping it in /var doesn't make anything better.

  • Historically the Unix directory tree is split across two or more disks. The root disks has /etc, and /bin and more. A second disk is mounted as /usr and contains /usr/bin among other things. This is getting changed so that the directories with static data on the root disk get moved to /usr and replaced with symlinks (e.g., /bin -> /usr/bin). After that, the root disk only has a minimal amount of variable data, such as /etc. Thus, almost the entire operating system is on a read-only disk, which has benefits for reliability, and can allow sharing it between, say, virtual machines or containers.

    This seems to a remarkably controversial change. Change is scary.

How did the Unix directory tree grow into the sprawling ent that it is today? It happened mostly through incremental change, not by design. You could almost say it happened organically.

For example, the /usr directory was originally, in the early 1970s, meant for home directories. "Usr" is short for "user" in various parts of Unix. What happened was the the Unix developers had two disks in their machine, one for the system, the other for user home directories. They ran out of disk space on the system disk, and fixed this by moving parts of the system to the user disk, eventually crowding out users from /usr to /home. As a result we now have the system spread on the "root disk", and the "user disk" (/usr).

Posted Wed Jan 11 12:00:06 2017 Tags:
Daniel Silverstone Give credit where credit is due

If you've been lucky enough to write some software worth publishing, and then super-lucky enough to have others like it enough to use it; and then ultra-mega lucky enough to have someone to like it enough to send you a patch for it; then you may be lucky enough to need to have a credits file.

When people contribute to your projects, it's only polite to make a little note so that they can see that their contribution was appreciated. Some of my projects even credit my employer when I've been fortunate enough to be allowed to write some F/LOSS at work.

The same applies if you "borrow" code from another project. Be sure to comply with their licensing terms, and even if they don't require it, it's only polite to thank them for their work. So for your homework, I'd like you to go back to any software you've written which has received patches from others and be sure to credit your contributors clearly and fully.

Posted Wed Jan 18 12:00:06 2017

We previously spoke of the FHS directory standard.

This includes important rules for where a program should store its data. This is particularly important when writing your programs as you need to know where you should write your program state to.

While this answers where you should write your state to for programs that run system-wide, it does not help for per-user programs.

The traditional unix approach for this has been dotfiles so each program leaves hidden files all over the user's home directory.

This is sub-optimal since the directories are hidden by default, which solves not cluttering the file listing in your home directory at the cost of merely hiding the clutter rather than structuring it.

To plug this gap the XDG directory standard was designed.

Rather than it specifying exactly which directories should be used, it specifies an environment variable which descibes which directory to use and a default if the environment variable is not defined.

Variable Default FHS equivalent Purpose
XDG_CONFIG_HOME ~/.config /etc Program configuration files.
XDG_CACHE_HOME ~/.cache /var/cache Non-essential cached data.
XDG_DATA_HOME ~/.local/share /usr/share Data files.
XDG_RUNTIME_DIR /var/run Program state for current boot.

The standard also defines XDG_DATA_DIRS and XDG_CONFIG_DIRS, but they are about where to read system-wide config files from.

The rules are meant to be simple enough that you can implement it yourself, but there are some helper libraries available.

If you're writing python there is PyXDG. If you're writing C there's implementations in GLib, or a 3-clause BSD licensed implementation in chck.

If you're interested in more standards for where to put files, a relevant XDG user directory standard exists, which lists more variables in ${XDG_CONFIG_HOME}/user-dirs.dirs for directories like the default place to download files to.

Posted Wed Jan 25 12:00:07 2017 Tags: