$ git init myproject
$ cd myproject
$ ...
$ ./profit

Thinking of a name, making the directory, putting it under revision control, firing up your editor -- all this is easy. Imagining the endgame where your users get to use the result of all your work is a little harder. What's really hard is all that ... in the middle. Deciding what bits of your project go where can be a toughie though, unless you're lucky enough to already know where things ought to go, it can be a bit of a nightmare.

Fortunately for most generalised kinds of projects there will be one or more recommended layouts for your codebase, just like there's a standard for the over-all filesystem heirarchy. Let's explore a few of these...

Generic C/C++ project

Typically a generic C/C++ project will have a hierarchy akin to:

myproject/
  Makefile / configure.ac / CMakeLists / Whatever
  src/
    your C and C++ files go here, and headers private to the program
  lib/
    C and C++ files, for libmyproject go here
  include/
    header files for libmyproject go here

This isn't strict, and more complex projects will usually deviate in interesting ways from this, but it's a good start for a basic project.

Python Modules and programs

Python modules and programs often have a standardised shape too, in part because of the utilities which exist for preparing them:

myproject/
  setup.py
  myprojectlib/
    all your shiny library files in here, __init__.py etc.
  myproject

Again, while not strict, the python distutils define a number of expectations which when met make life easier for the budding pythonista.

Java

There are a number of 'standard' build systems for Java, but I'll show you the layout I've learned which seems to be common for gradle built apps of the kind I've played with, with IntelliJ IDEA as the IDE.

myproject/
  build.gradle settings.gradle gradlew etc. (gradle stuff)
  gradle/
    in here goes some magic to make gradle work
  myproject.ipr etc (IntelliJ stuff)
  src/
    main/
      java/
        myproject/
          Your java files go here
      resources/
        Your resources go here

Conclusions

Clearly the above do not encompass every kind of project, nor do they accurately represent all instances of the generalised kinds of project they claim to be; however they are good starting points if you are beginning a project and you're not sure quite how to lay it out. In a lot of cases, the general layout is dictated by the standardised tools for building such projects, and in that case it would behoove you to follow the standard for such tools as you choose to use.

Of course, which build tools you choose is another aesthetic (and functional) decision, and we'll perhaps cover that in a future article.

Ultimately what matters is that, for your project, your layout is logical and does not introduce difficulty when locating parts of the source. If necessary you can have a document in your source tree explaining where things are. So long as you stick to your layout consistently, noone will be overly upset if things are not quite as they expected.


P.S. I tried to include Go here, but it just made me feel sick so I gave up, sorry.

P.P.S. No, I never claimed to be non-partisan here.

Posted Wed Jun 3 11:00:08 2015
Will Holland Jargon

How often have you found yourself searching for that shell command you have used 755 times before? What makes the incantation so hard to remember? Maybe it involves a string of seemingly random characters, something a lot of older Unix packages are guilty of, or maybe the choice of terminology is poor.

Software, like many geekdoms, is full of pop-culture references, acronyms, made-up words, and puns. This tangled web of jargon can lead to confusion and drive new-comers away. If you are writing a commandline package then its name, subcommands, config, and all other terminology needs to be simple and descriptive.

An example of a package which uses poorly chosen terminology is the version control system Git. If you have ever used Git then you will have committed changes. These changes must be staged, which is done by adding the change to the index; to view the index the command git status is used. Stage, add, index and status. Four terms have been introduced to refer to what is in practice a straight forward concept. In the source code these four concepts are probably distinct but to the user there is no need to introduce so many when fewer, more carefully chosen, terms could have done the job.

When deciding on a naming scheme there are a few things to bear in mind. First and foremost is to make the names easy to understand. Ensure that people have a good idea of how a concept fits into the project from its name alone. A technique commonly used for choosing terms is is to pick a metaphor. Metaphors are useful because they can show how terms relate to each other as well as doing a lot of the work for you when you need to find names for new concepts. For example maybe you are writing an image editor: the metaphor could be painting, the tool to add some pixels might be called the paint brush, the tool to remove pixels could be paint-stripper, and so on.

Choosing a naming scheme is not easy, but thankfully a lot of areas of software have pre-existing naming scheme norms. For example if you go to a website and it talks about comment then you have a good idea of what it means. If you avoid well established norms it can even lead to additional confusion.

Another point, worth noting, is to consider how words are pluralised. There do exist projects where people have been confused by the plural of a word not being obvious. For example if you have a script which creates pictures of sea-creatures, then to generate an octopus you might run ./sea-creatues --octopus, and to generate a picture of more than one octopus maybe ./sea-creatures --octopodes. Although octopodes is a real word it is not obviously the plural of octopus. Try to choose words which pluralise simply in English (ie. by appending the letter s).

Next time you write a piece of software, put some thought into the terminology: is it easy to understand or is it jargon? Using puns and pop-culture references rather than a simple metaphor may amuse a minority of people but it is likely to put others off.

Posted Wed Jun 10 11:00:09 2015 Tags:

systemd has a few services that are mostly just about providing a common interface to information about the current machine, or other machines.

Daemons for recording information

There's hostnamed(8), localed(8) and timedated(8) for providing a wirtable D-Bus interface to the system's hostname, locale and time respectively; and machined(8), which provides information about the "chassis type" or "form factor".

It may sound wasteful to have an entire server just to make this information available over D-Bus, but when a server is waiting for requests it is often paged-out, meaning it's only consuming a small amount of memory.

However, this wasn't good enough for the systemd developers, so these services are D-Bus activated, which means that they'll only be started when something makes a request of them. Combined with the fact that these services will exit when idle, and they don't even need to consume that small amount of memory any more.

One user of hostnamed(8) is the myhostname NSS module, which if nsswitch.conf is configured correctly, allows the addresses of the curent machine to be resolved without hard-coding them in the hosts file.

machined supporting containers

machined(8) also manages information about other machines, though this stretches the definition of "machine" a little, since it means the host machine, containers and virtual machines, rather than multiple physical machines on a network.

machined(8) is usually used indiectly by your virtual-machine manager or container manager, to provide a common interface for managing them with machinectl(1).

The usual management tasks aren't much more than just seeing which machines are online and stopping them if necessary, but if a machine is registered with machined(8), then other systemd management commands, such as systemctl(1) allow you to pass a machine name to the --machine option, to get it to operate on that machine instead of the host.

systemd comes with a container manager built-in, called systemd-nspawn(1), which has more full integration with machinectl(1), allowing the download of container images, configuring whether they start on-boot, and commands to log-in to the container and copy files in or out.

Posted Wed Jun 17 11:00:09 2015 Tags:
Daniel Silverstone Project organisation

There are many political systems which exist in the world. Most of us live in a democracy of some kind or another. However there are also autocratic regimes and communist societies. It is not my place to state that one or other of those approaches is "better" and indeed all of the above are used in FOSS projects of one kind or another.

While none of these examples are perfect, you could see OpenStack project or Debian as democracies since for the most part, changes (either in code in OpenStack's case, or project structure/policy in Debian's case) are voted on. An example of an autocratic project might be Python whose founder Guido van Rossum is by definition the project's Benevolent Dictator For Life. Projects whose governance more closely mirrors communism include CentOS which is essentially "by the users, for the users" and is very successful with it.

There is, however, one project organisational scheme which I am aware of, for which I cannot think of a political instance of outside of FOSS projects, and that is a do-ocracy. Since many FOSS projects are managed by small teams, or even individuals, a doocratic approach is often taken. Maintainers, while they will be wary of patches "thrown over the wall", will often be grateful for work done on their behalf and unbidden. In such a project, simply by virtue of doing something do you become authorised to do more.

The larger a project is, the more likely it is that the management of the project will be a blend of all of the above, along with its own unusual rules, tics, tweaks and confusion points. Part of the fun of contributing to FOSS, for me at least, is in learning how to fit into such a wide variety of organisations productively and enthusiastically. No matter if it interests you or not, understanding a project's organisational structure will aid you greatly in successfully participating in, and changing, a project.

Posted Wed Jun 24 11:00:07 2015