Recent changes to this wiki:

creating tag page tags/utmp
diff --git a/tags/utmp.mdwn b/tags/utmp.mdwn
new file mode 100644
index 0000000..d36612b
--- /dev/null
+++ b/tags/utmp.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged utmp"]]
+
+[[!inline pages="tagged(utmp)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/setsid
diff --git a/tags/setsid.mdwn b/tags/setsid.mdwn
new file mode 100644
index 0000000..c4d890b
--- /dev/null
+++ b/tags/setsid.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged setsid"]]
+
+[[!inline pages="tagged(setsid)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/pgroup
diff --git a/tags/pgroup.mdwn b/tags/pgroup.mdwn
new file mode 100644
index 0000000..de0fcfe
--- /dev/null
+++ b/tags/pgroup.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged pgroup"]]
+
+[[!inline pages="tagged(pgroup)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/wtmp
diff --git a/tags/wtmp.mdwn b/tags/wtmp.mdwn
new file mode 100644
index 0000000..4c5a977
--- /dev/null
+++ b/tags/wtmp.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged wtmp"]]
+
+[[!inline pages="tagged(wtmp)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/job
diff --git a/tags/job.mdwn b/tags/job.mdwn
new file mode 100644
index 0000000..69c8fb3
--- /dev/null
+++ b/tags/job.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged job"]]
+
+[[!inline pages="tagged(job)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/procrun-3-unix-sessions.mdwn b/posts/procrun-3-unix-sessions.mdwn
new file mode 100644
index 0000000..3c68360
--- /dev/null
+++ b/posts/procrun-3-unix-sessions.mdwn
@@ -0,0 +1,112 @@
+[[!meta date="Wed, 26 Apr 2017 12:00:07 +0000"]]
+[[!meta title="Is your process running 3 - UNIX sessions"]]
+[[!meta author="Richard Maw"]]
+[[!tag process job pgroup daemon kill setsid session utmp wtmp]]
+
+We previously discussed issues with using PIDfiles.
+
+One issue we encountered was that we need a way to handle multiple processes.
+
+Process groups
+===
+
+If you've ever started a program in the background in a shell
+you might have noticed it gave you a "Job ID" to refer to it
+rather than a process ID.
+
+This is not just to give you a memorable number for each task,
+but because jobs may contain multiple processes,
+which is how a pipeline of multiple processes may be a single job.
+
+This is accomplished in Linux and traditional UNIXes
+with the [setpgrp(2)][] system call
+which assigns a new process group to a process
+which will be inherited by its subprocesses.
+
+This entire process group may be killed
+by passing the negation of the process group ID
+to the [kill(2)][] system call.
+
+A process may only be part of one process group though,
+so if you have processes that may call [setpgrp(2)][] themselves
+then it is not possible to use process groups
+to manage terminating a whole process tree of a service.
+
+UNIX Sessions
+===
+
+You may be wondering how anything can possibly work
+if you can't use process groups to track a user's processes.
+
+The answer to this is that UNIX has a concept of sessions.
+
+Every process is a part of a session,
+and each session has a "controlling TTY",
+which can be accessed via `/dev/tty`.
+
+When a process creates a new session with [setsid(2)][]
+it becomes the session leader.
+
+If the session leader process is terminated
+then the entire session receives the `SIGHUP` signal,
+which by default terminates the process.
+
+The controlling tty was traditionally a [virtual terminal][]
+which emulates the old teletype terminals on modern computers.
+[Terminal windows in graphical interfaces][xterm] use [pseudo terminals][],
+which could be used to use sessions
+for grouping processes that don't belong to a device.
+
+This is typically done by [getty][] and [login(1)][], [terminal emulator][xterm]
+or [sshd][],
+which also update [utmp(5)][] to include the controlling TTY and session ID,
+to track the current active sessions.
+
+There are a number of issues with using UNIX sessions for tracking processes.
+
+1.  [utmp(5)][] is an awkward interface,
+    requiring multiple processes to access a single file
+    without trampling over each other,
+    requiring file range locking,
+    which can't be done portably and in a thread-safe manner.
+
+    I consider this to be analogous to [/etc/mtab][mtab],
+    which was an old, manually maintained file,
+    which had to be replaced with a more reliable, kernel-provided interface.
+
+2.  [setsid(2)][] describes sessions and process groups
+    as a strict two-level hierarchy.
+
+    The implication of this is that any process can escape with [setsid(2)][],
+    so bypassing [mtab][]
+    and inspecting the contents of the `sessionid` flie in `/proc` won't work.
+
+3.  Escaping session cleanup is by necessity a well-documented procedure,
+    since traditional daemons are started by detaching from the current session
+    rather than asking the [init][] process to start the daemon.
+
+    See [nohup(1)][] for details about how to escape session cleanup.
+
+Conclusion
+===
+
+The traditional UNIX system calls came from a time
+when it was believed you could trust programs to be well written and benign.
+
+We do not live in this world,
+so we need a better approach to track which processes we run on our computers,
+which we will discuss in a future article.
+
+[kill(2)]: http://man7.org/linux/man-pages/man2/kill.2.html
+[setsid(2)]: http://man7.org/linux/man-pages/man2/setsid.2.html
+[setpgrp(2)]: http://man7.org/linux/man-pages/man2/setpgrp.2.html
+[virtual terminal]: https://en.wikipedia.org/wiki/Virtual_terminal
+[pseudo terminals]:  https://en.wikipedia.org/wiki/Pseudoterminal
+[xterm]: https://en.wikipedia.org/wiki/Xterm
+[getty]: https://en.wikipedia.org/wiki/Getty_(Unix)
+[login(1)]: http://man7.org/linux/man-pages/man1/login.1.html
+[sshd]: https://linux.die.net/man/8/sshd
+[utmp(5)]: http://man7.org/linux/man-pages/man5/utmp.5.html
+[mtab]: https://en.wikipedia.org/wiki/Mtab
+[init]: https://en.wikipedia.org/wiki/Init
+[nohup(1)]: http://man7.org/linux/man-pages/man1/nohup.1.html

publishing
diff --git a/posts/motivation.mdwn b/posts/motivation.mdwn
new file mode 100644
index 0000000..f72f425
--- /dev/null
+++ b/posts/motivation.mdwn
@@ -0,0 +1,56 @@
+[[!meta date="Wed, 19 Apr 2017 11:24:16 +0000"]]
+[[!meta title="Find your motivation"]]
+[[!meta author="Daniel Silverstone"]]
+[[!tag motivation]]
+
+A [while ago][] I wrote about ensuring that you know why you're writing
+something in order that you keep focussed on that goal while you code.  My
+focus at that point was on the specific project you were looking to undertake,
+but it may behoove us to look at the wider picture from time to time.
+
+> What motivates you?
+> ===================
+
+It's important that you can answer this question, ideally without hesitation or
+backtracking.  For each of us the answer will be different, and noone's answer
+is any less "right" than anyone elses.  For myself, it took several years to be
+in a position to answer the question confidently, quickly, and consistently.
+That's not to say that my answer won't change in the future, but at least for
+now I know what motivates me and how that manifests in my day-to-day hacking.
+
+I have had a word with [Richard][] and he has explained his motivation to me,
+and so for your perusal and criticism, here's what motivates us both..
+
+Daniel
+------
+
+For me, the primary motivation for writing free software is that I enjoy making
+it possible for other people to achieve things.  I am, as it were, an "enabler"
+or "facilitator".  This manifests itself in an interest in processes,
+meta-programming, and tooling.  I find myself writing libraries, services, and
+test tooling; and I enjoy reading papers and architecture designs, thinking of
+new ways to solve old problems, and novel problems to solve.  (And of course,
+I write articles on this here blog :-) )
+
+Richard
+-------
+
+> My motivation in general is to learn something such that it can be applied to
+> something which in some way may be construed as to the betterment of society.
+> Or indeed those things which may improve society directly.  In the
+> free-software world, this has manifested in the topic of reliability and also
+> freeing people from vendor lock-in.
+
+(* note, I kinda paraphrased what Richard said)
+
+Homework
+========
+
+You didn't think I'd let you get away with no homework this week did you?  Hah!
+I'd like you to sit down, consider your motivation in the free software world
+and a few ways in which that manifests into projects you work on or with.  If
+you're feeling super-enthusiastic about it, why not post a comment on this post
+and share your motivation with the rest of us?
+
+[while ago]: /posts/truism-1-if-you-dont-know-why/
+[Richard]: /bio/richard/

creating tag page tags/kill
diff --git a/tags/kill.mdwn b/tags/kill.mdwn
new file mode 100644
index 0000000..fa993d4
--- /dev/null
+++ b/tags/kill.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged kill"]]
+
+[[!inline pages="tagged(kill)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/pidfile
diff --git a/tags/pidfile.mdwn b/tags/pidfile.mdwn
new file mode 100644
index 0000000..306be28
--- /dev/null
+++ b/tags/pidfile.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged pidfile"]]
+
+[[!inline pages="tagged(pidfile)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/proc
diff --git a/tags/proc.mdwn b/tags/proc.mdwn
new file mode 100644
index 0000000..03ad548
--- /dev/null
+++ b/tags/proc.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged proc"]]
+
+[[!inline pages="tagged(proc)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/procrun-2-pidfiles.mdwn b/posts/procrun-2-pidfiles.mdwn
new file mode 100644
index 0000000..710b3d0
--- /dev/null
+++ b/posts/procrun-2-pidfiles.mdwn
@@ -0,0 +1,196 @@
+[[!meta date="Wed, 19 Apr 2017 11:20:30 +0000"]]
+[[!meta title="Is your process running 2 - Nobody does pidfiles right"]]
+[[!meta author="Richard Maw"]]
+[[!tag process systemd daemon pidfile proc kill flock]]
+
+Previously we lamented that we had to read the `cmdline` for every process
+to work out whether the program we want to run is already running,
+and that we'd like to be able to look up a name to see if it's running.
+
+The traditional way to do this
+is to write the [process identifier (or PID)][PID]
+to a file called named after your program,
+such as `/var/run/$PROGNAME.pid`.
+
+This is an obvious solution, that you will probably have seen before,
+and before [systemd][] and [upstart][] became popular,
+was _the_ way you handled running services in Linux.
+
+How people do pid files wrong
+===
+
+On the surface just writing the [PID][] looks like a good idea,
+since you can use the presence of the file to tell that the process is running
+or read the contents of the file to see which [PID][] it's using
+so you can [kill(2)][] the process.
+
+This however is less reliable than parsing files in [procfs][],
+since if you're reading from `/proc` you know it's *always* up to date
+since it's provided by the kernel's process table.
+
+There are no such guarantees from the [PID][] file.
+Your process could terminate abnormally and not clean it up.
+
+Be extra cautious with [PID][] files not stored in `/run` or `/var/run`,
+since there is no guarantee that they weren't left over from a previous boot.
+
+Reliability can be improved
+by checking whether the [PID][] from the file is in use,
+by running [kill][kill(2)]`(pid, 0) == 0` in C,
+or [kill][kill(1)]` -0 $pid` in shell,
+since this will evalutate to true if that process is running.
+
+This is not yet reliable though,
+since while killing with signal 0 will tell you
+whether a process with that [PID][] is running,
+you can't tell if it's the *correct* process,
+since [PID][]s get reused fairly frequently.
+
+For [PID][] files to be reliable
+you need some way to guarantee that the [PID][] in the file
+is actually referring to the correct process.
+
+You need some way of tying the lifetime of the process to the file.
+
+When a process terminates all the file descriptors it had open are closed,
+so we can know that the contents are valid if a process is holding it open.
+
+You may be tempted to use [lsof][lsof(1)]` -Fp -f -- /var/run/$PROGNAME.pid`,
+or parsing [procfs][] manually to determine whether a process is using the file,
+but this is awkward for the same reason
+as not parsing the output of the [ps(1)][] command to tell whether it's running.
+
+We need to be able to do something to the file descriptor
+that will have an observable effect through other file descriptors to that flie.
+
+The solution to this is to take a lock on the file.
+
+You may see some programs use [fcntl(2)][] with `F_SETLK`.
+This is tempting because `F_GETLK` will include the [PID][] in the result
+instead of requiring the service to serialise and write the [PID][] to the flie
+and the checking process having to read and parse the file.
+
+`F_SETLK` should be avoided because the lock is removed
+when *any* file descriptor to that file owned by that process is closed,
+so you need to be able to guarantee
+that neither any of your code not any other code you use via a library
+will ever open that [PID][] file again
+even if your process normally opens arbitrary files by user input.
+
+So rather than using [fcntl(2)][] with `F_SETLK`,
+use [flock(2)][] with `LOCK_EX`, or [fcntl(2)][] with `F_WRLK`
+to take a write or exclusive lock on the file,
+and test whether the contents are live by trying to take a read or shared lock,
+with [flock(2)][]'s `LOCK_SH`, or [fcntl(2)][]'s `F_RDLK`.
+
+If you succeed at taking a read lock then the contents of the file aren't valid
+and the service isn't running.
+
+Implementing this logic can be awkward,
+fortunately if you're writing a C program
+you can use the [libbsd-pidfile][] functions.
+
+It's the wrong tool for the job
+===
+
+Setting it up correctly is tricky
+---
+
+The [libbsd-pidfile][] functions will handle locks correctly,
+but as permissively licensed as it is,
+it is not always available, perticularly if you're not writing a C program.
+
+A brief survey of the top results for python libraries for handling PID files
+resulted in [pid][trbspid], [python-pidfile][] and [pidfile][pypidfile].
+
+[python-pidfile][] does not keep the file open or take a lock.
+[pidfile][pypidfile] only improves by taking the lock and holding it.
+[pid][trbspid] checks the validity and sets correct permissions on the file.
+None of them have a mechanism to safely retrieve the [PID][] from the file.
+
+So if you have to do it yourself,
+you'll need to do the following:
+
+1.  open with `O_CREAT` to make the lock file if it doesn't exist.
+2.  Try to non-blocking take a shared lock on the file.
+    If you fail it's running,
+    and depending on whether you want to end it, replace it or leave it,
+    you should either instruct it to
+    terminate and continue, exit, or terminate then exit.
+    If you succeed then it's not running.
+3.  If you want to start a new service or replace it,
+    upgrade your shared lock to an exclusive lock **with a timeout**.
+    If you take a blocking lock then your processes will deadlock
+    waiting for their turn to start the service,
+    and if you take a non-blocking lock then
+    if your process is pre-empted by another process between trying to lock
+    and releasing the shared lock,
+    then you could end up with every process exiting.
+4.  If you took the lock replace the contents of the file with your [PID][],
+    if you timed out unlock the file or exit.
+
+The above is already complicated and still doesn't handle edge-cases,
+such as another process trying to start
+between taking the lock and writing the [PID][],
+which [libbsd-pidfile][] handles with a retry loop.
+
+It also doesn't handle the file being [unlink][unlink(2)]ed while starting,
+which would cause you to have multiple services running.
+
+[libbsd-pidfile][] doesn't take the shared lock then upgrade,
+so if many processes often want to know whether it's running,
+then they would be unnecessarily contesting the lock.
+
+Views of PIDs are not universal
+---
+
+[Process namespaces][pidns] exist.
+
+While a single process' view of its [PID][] does not change,
+other processes can see it with different [PID][]s
+since processes in sub-namespaces are viewable in multiple namespaces
+and have a different [PID][] in each namespace.
+
+You can detect whether the process is running by whether locking fails,
+but you can't use the contents to terminate it,
+since the PID it knows it to be is different from the PID you can reach it with,
+unless you use [a restricted system call][setns(2)]
+to enter the namespace of the process
+in order to terminate it.
+
+[fcntl(2)][]'s `F_GETLK` would return the [PID][] correctly,
+but is unreliable in other areas.
+
+Services can span more than one process
+---
+
+The PIDfile model works acceptably if your service only runs in one process,
+but some programs spawn helper processes
+and terminating the lead process may not tidy up all the helper processes.
+
+There are ways to have the termination of one process
+cause the termination of others,
+however this just moves the problem of "how do you know your process is running"
+to the supervising process without solving the general problem.
+
+So, we know that PID files aren't necessarily the right tool for the job,
+in the next article we explore some alternatives.
+
+[PID]: https://en.wikipedia.org/wiki/Process_identifier
+[systemd]: https://www.freedesktop.org/wiki/Software/systemd/
+[upstart]: http://upstart.ubuntu.com/
+[kill(2)]: http://man7.org/linux/man-pages/man2/kill.2.html
+[procfs]: http://man7.org/linux/man-pages/man5/proc.5.html
+[flock(2)]: http://man7.org/linux/man-pages/man2/flock.2.html
+[fcntl(2)]: http://man7.org/linux/man-pages/man2/fcntl.2.html
+[libbsd-pidfile]: https://manpages.debian.org/testing/libbsd-dev/pidfile.3.en.html
+[pidns]: http://man7.org/linux/man-pages/man7/pid_namespaces.7.html
+[unlink(2)]: http://man7.org/linux/man-pages/man2/unlink.2.html
+[setns(2)]: http://man7.org/linux/man-pages/man2/setns.2.html
+[kill(2)]: http://man7.org/linux/man-pages/man2/kill.2.html
+[kill(1)]: http://man7.org/linux/man-pages/man1/kill.1.html
+[ps(1)]: http://man7.org/linux/man-pages/man1/ps.1.html
+[lsof(1)]: http://man7.org/linux/man-pages/man1/lsof.1.html
+[trbspid]: https://github.com/trbs/pid/

(Diff truncated)
Tweak my bio
diff --git a/bio/daniel.mdwn b/bio/daniel.mdwn
index 5381e8e..6e281ce 100644
--- a/bio/daniel.mdwn
+++ b/bio/daniel.mdwn
@@ -1,6 +1,6 @@
 # Daniel Silverstone
 
-Daniel Silverstone is a Senior Software Developer at [Codethink] and was
+Daniel Silverstone is a Solution Architect at [Codethink] (whatever that means) and was
 fundamental to the original concept of [Baserock].  Daniel studied computer
 science with cognitive sciences at [University College London][ucl] for a year before
 giving up and going into games development.

creating tag page tags/testing
diff --git a/tags/testing.mdwn b/tags/testing.mdwn
new file mode 100644
index 0000000..8507783
--- /dev/null
+++ b/tags/testing.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged testing"]]
+
+[[!inline pages="tagged(testing)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/workflow
diff --git a/tags/workflow.mdwn b/tags/workflow.mdwn
new file mode 100644
index 0000000..4028013
--- /dev/null
+++ b/tags/workflow.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged workflow"]]
+
+[[!inline pages="tagged(workflow)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/scaffolding
diff --git a/tags/scaffolding.mdwn b/tags/scaffolding.mdwn
new file mode 100644
index 0000000..5a749b2
--- /dev/null
+++ b/tags/scaffolding.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged scaffolding"]]
+
+[[!inline pages="tagged(scaffolding)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/packaging
diff --git a/tags/packaging.mdwn b/tags/packaging.mdwn
new file mode 100644
index 0000000..7b690ab
--- /dev/null
+++ b/tags/packaging.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged packaging"]]
+
+[[!inline pages="tagged(packaging)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/new-project
diff --git a/tags/new-project.mdwn b/tags/new-project.mdwn
new file mode 100644
index 0000000..1bf0368
--- /dev/null
+++ b/tags/new-project.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged new-project"]]
+
+[[!inline pages="tagged(new-project)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/scaffolding.mdwn b/posts/scaffolding.mdwn
new file mode 100644
index 0000000..ecac78a
--- /dev/null
+++ b/posts/scaffolding.mdwn
@@ -0,0 +1,63 @@
+[[!meta date="Wed, 05 Apr 2017 11:00:06 +0000"]]
+[[!meta title="New project? Start with the scaffolding"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag new-project project scaffolding testing packaging workflow]]
+
+So you're starting a new project. You have an idea for something, and
+a vision for how to implement it. What should you do first?
+
+If you're like me, you burn to start with the interesting coding bit
+first. After a few hours of furious typing, you've done the fun bit,
+and you need to add all the boring bits: automated tests (unless you
+did TDD, and even then there may be missing tests), documentation, a
+manual page, a README, perhaps packaging (Debian, RPM, language
+specific), a CI project, a home page, etc. There's a lot of it. That
+is usually when my enthusiasm fades. A lot of my projects end up as a
+single file in `~/bin`.
+
+I call the boring bits scaffolding, and I've learnt that it pays to do
+them first. Not only is that often the only time I will actually do
+them, but once the scaffolding is in place, the fun bits are also more
+fun to do.
+
+If I have unit and integration test frameworks in place, adding
+another test is only a small incremental task. If I haven't got them
+in place (particularly for integration tests), I tend to postpone
+writing tests. No chance of TDD, whereas when I put in an integration
+test framework, I often to TDD at the program level, not just at the
+individual class and method level.
+
+Likewise, if I start by writing a manual page, adding another sentence
+or paragraph is easy. Also, writing a manual page, or any
+documentation, clarifies my thinking about what the program should
+actually do, and how it should be used. This makes writing the code
+more pleasant.
+
+Having CI in place from the beginning, means the tests I write get
+exercised from the start. This finds bugs earlier, even if I run my
+tests manually as well. CI never thinks "I can run test faster this
+one time by taking this clever shortcut".
+
+Of course, putting up a lot of scaffolding takes effort, and it's all
+wasted if you don't actually want to write the program after all.
+Sometimes what sounds like a brilliant idea on the last train home
+from a party, makes no sense at all in the morning. (Personal
+experience, ahem.)
+
+Thus it may make sense to start with a simple proof of concept, or
+prototype implementation to verify the soundness of you idea, then
+throw that away, and set up scaffolding, before you start writing
+production code.
+
+My latest personal project has a manual page, unit and integration
+tests, Debian packaging, a CI project, and a home page. I can install
+it and run it. It does't yet do anything useful. Before all this, I
+wrote a prototype to prove the idea I had, and threw that away. Some
+time this year I will start dropping in modules to actually do useful
+stuff, and that'll be easy, since I won't need to worry about all the
+boring bits. As soon as it does anything useful, I can also point
+friends at a Debian package and ask them to try my latest masterpiece.
+
+What's more, I have a better chance of getting them to help by writing
+a module they need themselves, since they just need to write the
+module and don't need to worry about the rest.

creating tag page tags/tty
diff --git a/tags/tty.mdwn b/tags/tty.mdwn
new file mode 100644
index 0000000..5692b84
--- /dev/null
+++ b/tags/tty.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged tty"]]
+
+[[!inline pages="tagged(tty)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/recording
diff --git a/tags/recording.mdwn b/tags/recording.mdwn
new file mode 100644
index 0000000..1c66a9c
--- /dev/null
+++ b/tags/recording.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged recording"]]
+
+[[!inline pages="tagged(recording)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/web
diff --git a/tags/web.mdwn b/tags/web.mdwn
new file mode 100644
index 0000000..22374a1
--- /dev/null
+++ b/tags/web.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged web"]]
+
+[[!inline pages="tagged(web)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/recording-your-terminal.mdwn b/posts/recording-your-terminal.mdwn
new file mode 100644
index 0000000..16bf473
--- /dev/null
+++ b/posts/recording-your-terminal.mdwn
@@ -0,0 +1,92 @@
+[[!meta date="Wed, 29 Mar 2017 11:00:06 +0000"]]
+[[!meta title="Sweetie! You're gonna be a movie star! - Recording your Terminal"]]
+[[!meta author="Daniel Silverstone"]]
+[[!tag tty recording web]]
+
+From time to time, you write something which you really want to show off.  If
+it's a GUI application then you might start with some good screenshots, or
+possibly even make a screen-capture video of you using the application, so that
+people can watch and marvel at your amazing creation.
+
+If your wonderful program happens to be a command-line tool, then such videos
+are often a little lackluster or simply overly-large for the information they
+convey.  It's better if the text being displayed is actually text and that it
+can be copy-pasted out (so others can follow along in their own terminals).  To
+that end, there is a number of tools for recording operations in a terminal and
+allowing that to played back.  I will mention a few of them here, but this is
+by no means an exhaustive list; and I shall endeavour to not recommend any
+specific option over any other.
+
+Asciinema
+---------
+
+Asciinema is a pretty slick project housed at <https://asciinema.org/> with its
+source available at <https://github.com/asciinema/asciinema> and
+<https://github.com/asciinema/asciinema-player>.  Asciinema is available in
+[Debian][] and other Linux distributions and works very well.  The Asciinema
+community provides a hosted system and also the tooling necessary to provision
+the playback of recordings on your own website providing you can honour the
+[GPLv3][].  The client application is written in Python and has few
+dependencies.
+
+You can see an example at: <https://asciinema.org/a/42383>
+
+showterm
+--------
+
+Like Asciinema, showterm provides a hosted service, over at
+<https://showterm.io/> and an open-source tool hosted at
+<https://github.com/ConradIrwin/showterm>.  Showterm's server-side is also
+open, but is focussed on a hosted experience so expect to be running up a rails
+app if you want to host it yourself.
+
+Showterm is written in Ruby and is under the MIT licence as is its server.  It
+also relies on `ttyrec` in some circumstances.
+
+You can see an example at: <https://showterm.io/7b5f8d42ba021511e627e>
+
+TermRecord
+----------
+
+If you're hoping for standalone HTML output, then TermRecord might be what
+you're after.  It can be found at <https://github.com/theonewolf/TermRecord>
+and is written in Python.  Unlike the other offerings, TermRecord produces
+entirely standalone output which doesn't depend on any other JavaScript or
+server resources.
+
+The licence terms for the TermRecord output include MIT licensed JavaScript,
+and font content under the Ubuntu Font License 1.0.
+
+You can see an example at: <http://theonewolf.github.io/TermRecord/figlet-static.html>
+
+TTYGIF
+------
+
+The TTYGIF project deserves an honourable mention because despite its output
+not being copy/pasteable; an animated GIF is much smaller than a video file
+would likely be.  TTYGIF is housed at <https://github.com/icholy/ttygif> and
+produces a simple animated [GIF][] of the terminal it is running in.  Since
+almost every web browser which isn't purely text-based offers animated GIF
+playback, the output of TTYGIF doesn't need any additional support.  No
+JavaScript players or complex plugins.
+
+TTYGIF is MIT licensed, but doesn't seem to place any particular licence on
+its output.  It's also written in C and needs a few dependencies to get itself
+going, one of which is a tty recorder anyway :-)
+
+You can see an example at: <https://camo.githubusercontent.com/acff4cc740350a784cb4539e501fcce1815329c0/687474703a2f2f692e696d6775722e636f6d2f6e76454854676e2e676966>
+
+Others
+------
+
+There are plenty of other options, including those which tend to only offer
+playback in a terminal themselves, such as `script` and `ttyrec`.  Your
+homework (You didn't think you'd got away with another week without homework
+did you?) is to have a play with some of the options listed here (or any
+others) you can find, and then perhaps comment on this posting showing what
+you've managed to get up to while creating a tty-cast to demonstrate your
+latest impressive terminal skills.
+
+[Debian]: https://www.debian.org/
+[GPLv3]: https://www.gnu.org/licenses/quick-guide-gplv3.en.html
+[GIF]: https://en.wikipedia.org/wiki/GIF

Fix weird paragraph formatting.
It looks reasonable as text,
but list items with multiple paragraphs get formatted differently,
and it switches only at the last item,
so give wider spacing to the rest too.
diff --git a/posts/procrun-1-parsing-proc.mdwn b/posts/procrun-1-parsing-proc.mdwn
index d2ea8f0..7496f00 100644
--- a/posts/procrun-1-parsing-proc.mdwn
+++ b/posts/procrun-1-parsing-proc.mdwn
@@ -84,16 +84,20 @@ but as usual caveats apply:
 1.  Not all processes have an initial executable.
     This symbolic link may be unreadable (fails with [errno][] of `ENOENT`)
     in the case of kernel threads.
+
 2.  It could be a program that has subcommands,
     one of which may be a long-running service (e.g. git-daemon),
     which you wouldn't want to fail to start
     just because a shorter operation with a different subcommand
     happened to be running at the same time.
+
 3.  This is unhelpful in the case of interpreted languages,
     since it is always the name of the interpreter
     rather than the name of the script.
+
 4.  The same program may be reachable by multiple file paths
     if the executable has been hard-linked.
+
 5.  If the program's executable may be removed while it is running,
     changing `exe` to append " (deleted)" to the file path.
 

Fix typo in executable
diff --git a/posts/procrun-1-parsing-proc.mdwn b/posts/procrun-1-parsing-proc.mdwn
index b25c0d8..d2ea8f0 100644
--- a/posts/procrun-1-parsing-proc.mdwn
+++ b/posts/procrun-1-parsing-proc.mdwn
@@ -94,7 +94,7 @@ but as usual caveats apply:
     rather than the name of the script.
 4.  The same program may be reachable by multiple file paths
     if the executable has been hard-linked.
-5.  If the program's execuable may be removed while it is running,
+5.  If the program's executable may be removed while it is running,
     changing `exe` to append " (deleted)" to the file path.
 
     If this file is then replaced

Fix up nonsense sentence
diff --git a/posts/procrun-1-parsing-proc.mdwn b/posts/procrun-1-parsing-proc.mdwn
index af1d864..b25c0d8 100644
--- a/posts/procrun-1-parsing-proc.mdwn
+++ b/posts/procrun-1-parsing-proc.mdwn
@@ -87,7 +87,8 @@ but as usual caveats apply:
 2.  It could be a program that has subcommands,
     one of which may be a long-running service (e.g. git-daemon),
     which you wouldn't want to fail to start
-    just because a long operation with a different subcommand.
+    just because a shorter operation with a different subcommand
+    happened to be running at the same time.
 3.  This is unhelpful in the case of interpreted languages,
     since it is always the name of the interpreter
     rather than the name of the script.

creating tag page tags/process
diff --git a/tags/process.mdwn b/tags/process.mdwn
new file mode 100644
index 0000000..13e0ef2
--- /dev/null
+++ b/tags/process.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged process"]]
+
+[[!inline pages="tagged(process)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/procrun-1-parsing-proc.mdwn b/posts/procrun-1-parsing-proc.mdwn
new file mode 100644
index 0000000..af1d864
--- /dev/null
+++ b/posts/procrun-1-parsing-proc.mdwn
@@ -0,0 +1,138 @@
+[[!meta date="Wed, 22 Mar 2017 12:00:08 +0000"]]
+[[!meta title="Is your process running 1 - Parsing proc and why you shoudn't"]]
+[[!meta author="Richard Maw"]]
+[[!tag process systemd daemon]]
+
+Sometimes it is necessary to leave a process running,
+performing some service in the background while doing something else.
+
+It would be redundant and possibly harmful
+to start a new one if it is already running.
+
+Ideally all programs would safely shut themselves down if already running,
+checking if it's running before starting
+only guarantees that it was runing when you checked,
+rather than that it is running when you need it.
+For most purposes though it is reasonable to check first.
+
+So how do we know if our service is running?
+
+You may have run [ps(1)][] before to see if a process is running,
+so you might naturally think this would be how to do it.
+
+This would of course fall into the trap of parsing the output of shell commands.
+Why should we write fragile code
+when [ps(1)][] is using a proper API to do it?
+
+The way this is accomplished
+is the [procfs][] [virtual file system][] traditionally [mounted][] at `/proc`.
+There is a subdirectory in this file system for each process
+listed by its process ID.
+
+We can list all directories that are processes by running:
+
+    find /proc -mindepth 1 -maxdepth 1 -name '[0-9]*'
+
+Inside each of these directories are files describing the process.
+
+Check `comm`
+===
+
+When you look at the output of [`ps`][ps(1)]
+it shows the name of the process,
+which is normally the base name of the file path
+of the executable that the process was started with.
+
+This is stored in the file in `/proc` called `comm`.
+
+So if the name of your program is "myprogram",
+you can find out if your program is running with the following command:
+
+    find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
+        '[ "$(cat "$1/comm")" = myprogram ] && echo Is running' - {} ';'
+
+I would recommend against checking if your program is running this way though,
+as processes may call themselves whatever they want,
+by writing the new name to `comm`.
+
+    $ cat /proc/$$/comm
+    bash
+    $ printf dash >/proc/$$/comm
+    $ cat /proc/$$/comm
+    dash
+
+This is often used by services that fork off helper processes
+to name the subprocesses after their role
+to make it easier for developers or sysadmins to know what they do.
+
+Check `exe`
+===
+
+The [procfs][] entry also includes
+the path of the executable the proccess was started from
+as a symbolic link.
+
+Thus if your program is installed at `/usr/bin/myprogram`
+then we can check whether it is running with:
+
+    find /proc -mindepth 1 -maxdepth 1 ! -name '*[^0-9]*' -type d -exec sh -c \
+        '[ "$(readink "$1/exe")" = /usr/bin/myprogram ] && echo Is running' - {} ';'
+
+This cannot be modified by the proces after it has started,
+but as usual caveats apply:
+
+1.  Not all processes have an initial executable.
+    This symbolic link may be unreadable (fails with [errno][] of `ENOENT`)
+    in the case of kernel threads.
+2.  It could be a program that has subcommands,
+    one of which may be a long-running service (e.g. git-daemon),
+    which you wouldn't want to fail to start
+    just because a long operation with a different subcommand.
+3.  This is unhelpful in the case of interpreted languages,
+    since it is always the name of the interpreter
+    rather than the name of the script.
+4.  The same program may be reachable by multiple file paths
+    if the executable has been hard-linked.
+5.  If the program's execuable may be removed while it is running,
+    changing `exe` to append " (deleted)" to the file path.
+
+    If this file is then replaced
+    then another process may have the same executable path
+    but an incompatible behaviour.
+    
+    This isn't even unusual if the name of the process is generic,
+    like "sh" or "httpd".
+
+So it's useless for interpreted programs
+and unreliable if the executable can be replaced.
+
+Check `cmdline`
+===
+
+It could be perfectly safe to run the same program multiple times
+provided it is passed different configuration.
+
+The `cmdline` file can be parsed to infer this configuration
+as a list of strings that are NUL terminated.
+
+A problem with this approach
+is the need to reimplement parsing logic
+*and* know for all command-lines whether it's appropriate to start another.
+
+This logic could be quite difficult,
+but you could add a parameter just for determining whether it is the same.
+
+This is far from ideal because:
+
+1.  Lookup time gets worse as your system has more processes running.
+2.  Processes can modify their command-line too,
+    so another process could arrange to have the same command-line,
+    and make this unreliable.
+
+Next time we are going to look at a better use for that parameter.
+
+[ps(1)]: http://man7.org/linux/man-pages/man1/ps.1.html
+[procfs]: http://man7.org/linux/man-pages/man5/proc.5.html
+[virtual file system]: https://en.wikipedia.org/wiki/Virtual_file_system
+[mounted]: https://en.wikipedia.org/wiki/Mount_(computing)
+[errno]: http://man7.org/linux/man-pages/man3/errno.3.html

creating tag page tags/it-worked-a-month-ago
diff --git a/tags/it-worked-a-month-ago.mdwn b/tags/it-worked-a-month-ago.mdwn
new file mode 100644
index 0000000..92b907e
--- /dev/null
+++ b/tags/it-worked-a-month-ago.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged it-worked-a-month-ago"]]
+
+[[!inline pages="tagged(it-worked-a-month-ago)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/bitrot
diff --git a/tags/bitrot.mdwn b/tags/bitrot.mdwn
new file mode 100644
index 0000000..fca2ca3
--- /dev/null
+++ b/tags/bitrot.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged bitrot"]]
+
+[[!inline pages="tagged(bitrot)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/bitrot.mdwn b/posts/bitrot.mdwn
new file mode 100644
index 0000000..04f3587
--- /dev/null
+++ b/posts/bitrot.mdwn
@@ -0,0 +1,61 @@
+[[!meta date="Wed, 15 Mar 2017 12:00:07 +0000"]]
+[[!meta title="What is this disgusting smell? On software bit rot"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag bitrot it-worked-a-month-ago]]
+
+[Bit rot]: https://en.wikipedia.org/wiki/Software_rot
+
+[Bit rot][], specifically the phenomenon of software working less well
+even if it hasn't changed, is annoying, but a fact of life. There
+might be a thing or two you can do to make it happen less.
+
+Examples from your author's personal experience from the past year:
+
+* Cloud provider changes the default username on base images from
+  `ec2-user` to `debian`, requiring simple changes needed in many
+  places.
+* Cloud provider upgrades their virtualisation platform, which
+  introduces a new API version, and breaks the old version. All API
+  using automation needs upgrading.
+* Configuration management software introduces a new feature
+  (`become`), and deprecates the old corresponding feature (`sudo`).
+  Simple changes, but in many places.
+* Configuration management software breaks the new feature (can no
+  longer switch to an unprivileged user to run shell script snippets),
+  requiring more complicated changes in several places (run shell as
+  root, invoke sudo explicitly).
+* Author's software depends on enterprise-grade software for a
+  specific service, which switches to requiring Oracle Java, instead
+  of OpenJDK. Author's software isn't fully free software anymore.
+
+Bit rot happens for various reasons. The most common reason is that
+the environment changes. For example, software that communicates over
+the network may cease to function satisfactorily if the other
+computers change. A common example is the web browser: even though
+your computer works just as well as before, in isolation, web sites
+use new features of HTML, CSS, and JavaScript, not to mention media
+formats, and web pages become bigger, and in general everything
+becomes heavier. Also, as your browser version ages, sites stop caring
+about testing with it, and start doing things that expose bugs in your
+version. Your web experience becomes worse every year. Your browser
+bit rots.
+
+There is no way to prevent bit rot. It is a constant that everything
+is variable. However, you can reduce it by avoiding common pitfalls.
+For example, avoid dependencies that are likely to change,
+particularly in ways that will break your software. An HTML parsing
+library will necessarily change, but that shouldn't break your
+software if the library provdes a stable API. If the library adds
+support for a new syntactic construction in HTML, your program should
+continue to work as before.
+
+You should be as explicit as possible in what you expect from the
+environment. Aim to use standard protocols and interfaces. Use
+standard POSIX system calls, when possible, instead of experimental
+Linux-specific ones from out-of-tree development branches. Sometimes
+that isn't possible: document that clearly.
+
+Have automated ways of testing that your software works, preferably
+tests that can be run against an installed instance. Run those tests
+from time to time. This will let you and your users notice earlier
+that something's broken.

publishing
diff --git a/posts/hiring-floss-people.mdwn b/posts/hiring-floss-people.mdwn
new file mode 100644
index 0000000..a360ad0
--- /dev/null
+++ b/posts/hiring-floss-people.mdwn
@@ -0,0 +1,49 @@
+[[!meta date="Wed, 08 Mar 2017 12:00:07 +0000"]]
+[[!meta title="What I look for in a F/LOSS applicant"]]
+[[!meta author="Daniel Silverstone"]]
+
+I have the dubious honour of being one of the people, at my place of work,
+charged with interviewing technical applicants.  Without giving the game away
+too much, I thought I might give a few hints for things I look for in a CV,
+and in the wider world, when considering and interviewing a candidate.
+
+First a little context - I tend to interview candidates who are applying for
+higher-level technical roles in the company, and I have a particular focus on
+those who claim on their CV to have a lot of experience.  I start by reading
+the cover letter and CV looking for hints of F/LOSS projects the applicant has
+worked with; either as a user or a developer.  I like it when an applicant
+provides a bitbucket, github or gitlab URL for their personal work if they have
+any; but I *really* like it when they provide a URL for their own Git server
+(as you might imagine).
+
+Once I have identified places on the Internet where I might find someone, I
+look to dig out their [internet ghosts][] and find out what they are up to in
+the wider F/LOSS world.  The best candidates show up in plenty of places, are
+easily found making nice commits which show their capability, and seem well
+spoken on mailing lists, fora, et al.  Of course, if someone doesn't show up
+on Internet searches then that doesn't count against them because to have the
+privilege of being able to work on F/LOSS is not something afforded to all; but
+if you *do* show up and you look awful it will count against you.
+
+[internet ghosts]: http://www.ibiblio.org/Dave/Dr-Fun/df9601/df960124.jpg
+
+Also remember, there's more ways to contribute than writing code.  I love it
+when I find candidates have made positive contributions to projects outside of
+just coding for them.  Help a project's documentation, or be part of mentoring
+or guide groups, and I'll likely be very pleased to talk with you.
+
+Beyond the _Internet Stalking_, I like to get my candidates to demonstrate an
+ability to compare and contrast technologies; so a good way to get on my good
+side is to mention two similar but conflicting capabilities (such as
+[Subversion][] and [Git][]) be prepared to express a preference between them,
+and be able to _defend_ that preference.
+
+[Subversion]: http://subversion.apache.org/
+[Git]: https://git-scm.com/
+
+Finally a few basic tips -- don't lie, dissemble, or over-inflate in your CV
+or cover-letter (I will likely find out) and don't let your cover letter be
+more than a single side of A4, nor your CV more than 2 sides of A4.
+
+If I ever interview you, and I find out you read this article, I will be most
+pleased indeed.  (Assuming you take on my recommendations at least :-) )

creating tag page tags/motivation
diff --git a/tags/motivation.mdwn b/tags/motivation.mdwn
new file mode 100644
index 0000000..9297e1c
--- /dev/null
+++ b/tags/motivation.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged motivation"]]
+
+[[!inline pages="tagged(motivation)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/facilitating.mdwn b/posts/facilitating.mdwn
new file mode 100644
index 0000000..ef24fed
--- /dev/null
+++ b/posts/facilitating.mdwn
@@ -0,0 +1,54 @@
+[[!meta date="Wed, 01 Mar 2017 12:00:07 +0000"]]
+[[!meta title="Facilitating is no less valuable than contributing"]]
+[[!meta author="Richard Maw"]]
+[[!tag contributing motivation]]
+
+FOSS projects are mostly developed on a volunteer basis.
+
+This makes the currencies by which they are developed: free time and motivation.
+
+Often times you have the free time, but not the motivation.
+Often this is not from feeling that the work isn't worth doing,
+but that you feel inadequate to do it.
+
+Don't be disheartened.
+There's plenty you can do that helps.
+
+1.  Just be there, whether in-person or online.
+
+    You can do whatever else you want while being there,
+    but it's encouraging to not be along in your endeavours.
+
+    You may even find some motivation of your own.
+
+2.  When others are talking about what they want to achieve,
+    respond enthusiastically.
+
+    It makes them more likely to follow-through and do so,
+    and in the very least makes them feel good.
+
+    This does risk making them feel worse if they never get around to it,
+    but sometimes that's sufficient to shame them into action later,
+    and other times it's sufficient to say "these things happen".
+
+3.  Engage in discussion about what others want to achieve.
+
+    It's extremely valuable for refining ideas,
+    so they can implement what they want to do better,
+    it keeps it fresh in their mind so motivation lasts longer,
+    and it leaves a clearer idea of what to do
+    so it may be completed before motivation runs out.
+
+4.  Mention what other people are doing to people who might be interested.
+
+    You could end up with anecdotes of other people thinking it's a cool idea,
+    which when relayed to people doing the work provides their own motivation.
+
+5.  Remind people of the successes they've had.
+
+    It makes people feel good about what they've already done,
+    and can put any issues they are currently struggling with into perspective.
+
+    Lars pointed out that
+    Yakking has published more than 180 articles at a rate of one per week!
+    We've managed to get this far, we can continue for a good while yet.

publishing
diff --git a/posts/be-careful-testing.mdwn b/posts/be-careful-testing.mdwn
new file mode 100644
index 0000000..abbba2e
--- /dev/null
+++ b/posts/be-careful-testing.mdwn
@@ -0,0 +1,56 @@
+[[!meta date="Wed, 22 Feb 2017 12:00:06 +0000"]]
+[[!meta title="Please be careful when you test"]]
+[[!meta author="Daniel Silverstone"]]
+
+We have [spoken][] [before][] about testing your software.  In particular we
+have mentioned how if your code isn't tested you can't be confident that it
+works.  Whe also spoke about how the technique of testing and the level at
+which you test your code will vary based on what you need to test.
+
+[spoken]: http://yakking.branchable.com/posts/truism-4-if-it-is-not-tested/
+[before]: http://yakking.branchable.com/posts/software-testing/
+
+What I'd like to talk about this time is about understanding the environment in
+which your tests exist.  Since "nothing exists in a vacuum" it is critical to
+understand that even if you write beautifully targetted tests, they still exist
+and execute within the wider context of the computer they are running on.
+
+As you are no doubt aware by now, I have a tendency to indulge in the hoary old
+developer habit of teaching by anecdote, and today is no exception to that.  I
+was recently developing some additional tests for [Gitano][] and exposed some
+very odd issues with one test I wrote.  Since I was engaged in the
+ever-satisfying process of adding tests for a previously untested portion of
+code I, quite reasonably, expected that the issue I was encountering was a bug
+in the code I was writing tests for.  I dutifully turned up the logging levels,
+sprinkled extra debug information around the associated bits of code, and
+puzzled over the error reports and debug logs for a good hour or so.
+
+[Gitano]: https://wiki.gitano.org.uk/about/what-is/
+
+Predictably, given the topic of this article, I discovered that the error in
+question made absolutely no sense given the code I was testing, and so I had to
+cast my net wider.  Eventually I found a bug in a library which Gitano depends
+on, which gave me a somewhat [hirsuite yak][] to deal with.  Once I had written
+the patch to the library, tested it, committed it, made an upstream release,
+packaged that, reported the bug in Debian, uploaded the new package to Debian,
+and got that new package installed onto my test machine - lo and behold, my
+test for Gitano ran perfectly.
+
+[hirsuite yak]: http://yakking.branchable.com/posts/debugging/
+
+This is, of course, a very particular kind of issue.  You are not likely to
+encounter this type of scenario very often, unless you also have huge tottering
+stacks of projects which all interrelate.  However you are likely to encounter
+issues where tests assume things about their environment without necessarily
+meaning to.  Shell scripts which use bashisms, or test suites which assume they
+can bind test services to particular well known (and well-used) ports are all
+things I have encountered in the past.
+
+Some test tools offer mechanisms for ensuring the test environment is "sane"
+for a value of sanity which applies only to the test suite in question.  As
+such, your homework is to go back to one of your well-tested projects and
+consider if your tests assume anything about the environment which might need
+to be checked for (outside of things which you're already checking for in order
+to build the project in the first place).  If you find any unverified
+assumptions then consider how you might ensure that, if the assumption fails,
+the user of your test suite is given a useful report on which to act.

Added a comment: You're welcome
diff --git a/suggestion-box/comment_3_2b3649f9e6a73cdf882009e9bc318b47._comment b/suggestion-box/comment_3_2b3649f9e6a73cdf882009e9bc318b47._comment
new file mode 100644
index 0000000..84169f9
--- /dev/null
+++ b/suggestion-box/comment_3_2b3649f9e6a73cdf882009e9bc318b47._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://liw.fi/"
+ avatar="http://cdn.libravatar.org/avatar/db5541dd6756dc8db2663b4785de58f490e13b301f4c421e20176bc7e6a5b8e4"
+ subject="You're welcome"
+ date="2017-02-18T15:19:31Z"
+ content="""
+Awwk, thank you ana. You're very kind.
+"""]]

Added a comment: Keep up the good work
diff --git a/suggestion-box/comment_2_e3608b27f5d4cfbff53f1ad11f637e5f._comment b/suggestion-box/comment_2_e3608b27f5d4cfbff53f1ad11f637e5f._comment
new file mode 100644
index 0000000..9459db0
--- /dev/null
+++ b/suggestion-box/comment_2_e3608b27f5d4cfbff53f1ad11f637e5f._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="ana@0d0ca1cf7a8ced0d9b9ac70c133c2ca672f5a077"
+ nickname="ana"
+ avatar="http://cdn.libravatar.org/avatar/a77ac42cd26e7b41c18bafaf4c3812af"
+ subject="Keep up the good work"
+ date="2017-02-18T13:49:45Z"
+ content="""
+Folks, I have been reading yakking for 3? 4? years now and your articles have been a very valuable source to refer to co-workers and friend when they were asking about some topics. And even for relearning about some topics myself. Thank you!
+"""]]

Fix typo (quantified)
diff --git a/posts/tips-from-fosdem.mdwn b/posts/tips-from-fosdem.mdwn
index ba64b8a..59921f8 100644
--- a/posts/tips-from-fosdem.mdwn
+++ b/posts/tips-from-fosdem.mdwn
@@ -35,7 +35,7 @@ and some specific cultural differences that often cause issue.
 
 It started out with a metric by which cultural attitudes
 about 6 supposed characteristics of culture
-can be quantised
+can be quantified
 so the difference between cultural attitudes can be measured.
 
 The speaker admitted that they weren't an expert on the topic

creating tag page tags/fosdem
diff --git a/tags/fosdem.mdwn b/tags/fosdem.mdwn
new file mode 100644
index 0000000..ded7c7f
--- /dev/null
+++ b/tags/fosdem.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged fosdem"]]
+
+[[!inline pages="tagged(fosdem)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/communities
diff --git a/tags/communities.mdwn b/tags/communities.mdwn
new file mode 100644
index 0000000..5437bbb
--- /dev/null
+++ b/tags/communities.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged communities"]]
+
+[[!inline pages="tagged(communities)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/tips-from-fosdem.mdwn b/posts/tips-from-fosdem.mdwn
new file mode 100644
index 0000000..ba64b8a
--- /dev/null
+++ b/posts/tips-from-fosdem.mdwn
@@ -0,0 +1,215 @@
+[[!meta date="Wed, 15 Feb 2017 12:00:08 +0000"]]
+[[!meta title="Things I learned at FOSDEM"]]
+[[!meta author="Richard Maw"]]
+[[!tag conference fosdem communities]]
+
+This year I was fortunate enough to attend [FOSDEM][].
+
+Since a [project I work on][gitano] recently made its [1.0 release][semver]
+and is scheduled to be part of the next [Debian stable][] release
+this means it will plausibly attract more attention.
+
+Being inexperienced in community engagement,
+I decided to attend various talks on this topic.
+
+[Building an accessible community][culture-accessible] ([video][culture-accessible-video])
+==========================================================================================
+
+This is mostly not relevant to [Gitano][]
+because it is about how to run a conference and handle accessibility.
+
+It was entertaining though,
+and a useful bit of advice, that little things
+like using gender neutral pronouns in text
+can help to foster an inclusive atmosphere.
+
+As a result I will try to proof read my writing
+in case I unintentionally included non-inclusive language.
+
+[Overcoming culture clash][culture-clash] ([video][culture-clash-video])
+========================================================================
+
+This was a talk about
+some theory about what kinds of cultural differences there are,
+and some specific cultural differences that often cause issue.
+
+It started out with a metric by which cultural attitudes
+about 6 supposed characteristics of culture
+can be quantised
+so the difference between cultural attitudes can be measured.
+
+The speaker admitted that they weren't an expert on the topic
+so I couldn't ask for clarification
+of how some characteristics differed
+since my impression was that at least three of them overlapped.
+
+More practical advice included:
+
+1.  Communities are built one person at a time.
+    So we should try to foster good relations with existing and new users.
+2.  Local support groups should be encouraged,
+    but should be helped to interact with the wider community
+    by inviting members to events and visiting them.
+3.  Get to know the cultural differences of local groups.
+4.  Avoid real-time (face to face or IRC) meetings.
+    Text is better than video calls,
+    and asynchronous is better than synchronous
+    since it is a lot easier to translate.
+6.  Plan events with awareness of religious and national holidays
+    so you're not accidentally excluding someone.
+    For example, FOSDEM is set during Chinese new year,
+    so can be problematic.
+7.  Don't be afraid to ask if people have issues,
+    but be aware that not wanting to impose is also a cultural value,
+    so they may attempt to appease unnecessarily.
+
+[Like the ants][culture-growing] (Growing communities)
+======================================================
+
+There was a talk from a communities manager from Google
+talking about how to foster a community without driving it.
+
+The gist is that it should emulate a hive-mind like ants,
+where rather than dictating direction
+you would provide feedback mechanisms to encourage what is wanted.
+
+This is not relevant to [Gitano][]
+since we will be part of any community that happens.
+
+[Open source is just about the source, isn't it?][culture-non-source] ([video][culture-non-source-video])
+=========================================================================================================
+
+This was a talk from a community manager
+about a bunch of non-code parts of a project to worry about.
+
+Trademarking
+------------
+
+First was trademark handling.
+
+For [Gitano][] we've started well by picking a name
+that is not already used for a git server
+and created a [logo][gitano-logo] that resembles no other git server.
+
+We will need to ask anyone who names their git server Gitano to rename though.
+
+Finding users
+-------------
+
+We need to go out and find potential users,
+rather than waiting for them to come to us.
+
+Talking about it on social media may help,
+and getting users to talk about it would help.
+
+As would submitting talks to relevant conferences.
+
+Larger projects can submit an article to a relevant journal
+since journalists are lazy and will print articles to fill space.
+
+I intend to speak about Gitano at more conferences as a result.
+
+Supporting users
+----------------
+
+Supporting users is essential,
+you don't know who might be a valuable contributor
+so be friendly to everyone.
+
+You can't always expect users to come to you, you need to go where they are,
+which may mean [subscribing to Stack Overflow][trello-so]
+to see if [Gitano][] is mentioned.
+
+Retaining contributors
+----------------------
+
+Non-coder contributors can be hugely valuable,
+since they provide support for other users
+and may be able to provide support in languages you don't understand.
+
+Retaining contributors depends heavily on how responsive you are.
+If you can provide automated feedback on code style etc. it helps.
+
+If a useful contributor is approaching burn-out,
+if you can arrange for them to be employed to do so then that's handy,
+this is not helpful for [Gitano][] since we're not a big foundation.
+
+There isn't much we can do before other contributors turn up or leave for this.
+
+Managing infrastructure
+-----------------------
+
+If infrastructure is required for development then upstream must provide it,
+since contributors are even less likely to be able to provide their own.
+
+If infrastructure is expensive then a tip jar helps.
+
+Try not to spread infrastructure across too many systems,
+or at least provide a landing page to locate everything.
+
+Have one place for recording canonical decisions,
+don't split them across mailing lists or wikis.
+
+For [Gitano][] we expect the canonical place for policy to be the wiki,
+so we're going to have a [policy page][trello-policy]
+and [link to all the infrastructure][trello-infra] from the wiki.
+
+All [Gitano][]'s infrastructure is either free
+or cheap and paid out of the lead developer's pocket,
+so we don't need to think about a tip jar yet.
+
+Expect to leave
+---------------
+
+Plan for your own exit from the project.
+Nothing lasts forever.
+
+It is helpful if you can centralise contact details
+so they can be changed more easily.
+
+For [Gitano][] we [plan to have a contact details page][trello-contacts]
+to centralise as much as we can,
+and avoid giving out the details of any particular person as a contact.
+
+Deciding on communications channels
+-----------------------------------
+
+Given the discussion of various contact channels
+and the importance of using appropriate ones
+I asked the speaker if she had any tips on how to evaluate which to use.
+
+I was recommended to decide based on:
+
+1.  Which is easy for your developers to use.
+2.  Which is easy for your contributors to use.
+
+    Ideally by asking existing users what they would prefer,
+    but otherwise an educated guess based on what the target users might want.
+
+In [Gitano][] we have opinionated developers who like mailing lists,
+IRC and RSS feeds,
+so to widen the support net
+we're going to add a [link to webchat][trello-webchat].
+
+[FOSDEM]: http://www.fosdem.org

(Diff truncated)
publishing
diff --git a/posts/internationalisation.mdwn b/posts/internationalisation.mdwn
new file mode 100644
index 0000000..a6deabc
--- /dev/null
+++ b/posts/internationalisation.mdwn
@@ -0,0 +1,37 @@
+[[!meta date="Wed, 08 Feb 2017 12:00:07 +0000"]]
+[[!meta title="Internationalisation and Other Fruit"]]
+[[!meta author="Will Holland"]]
+
+Any software that is used by more than a trivial number of people will at some
+point be used by people living in different countries, with different cultures.
+Different cultures have different conventions which might cover [date
+format][], [decimal mark][], currency, [names][], [units of
+measurement][], [spelling][] and of course, language (that's [natural][]
+language, not [programming][] language).
+
+The practice of building support for a variety of cultures into your software
+is called [internationalisation][], often referred to as [I18N][]. This is the
+practice of writing software that is independent of any particular culture.
+There is a closely relation topic [localisation][] (L10N). This can be thought
+of as taking an internationalised piece of software and localising it for one
+specific region.
+
+You should be careful to separate your internationalisation from your [business
+logic][].
+
+It is unlikely that you can cover every possible variation in this yourself.
+Luckily, there are several [tools][] designed to deal with the problem of
+internationalisation. These will be covered in a future article so stay tuned!
+
+[date format]: https://en.wikipedia.org/wiki/Date_format_by_country
+[decimal mark]: https://en.wikipedia.org/wiki/Decimal_mark
+[names]: https://www.w3.org/International/questions/qa-personal-names
+[units of measurement]: http://chartsbin.com/view/d12
+[spelling]: http://www.quickanddirtytips.com/education/grammar/why-are-british-english-and-american-english-different
+[natural]: http://www.webopedia.com/TERM/N/natural_language.html
+[programming]: http://www.webopedia.com/TERM/P/programming_language.html
+[internationalisation]: https://en.wikipedia.org/wiki/Internationalization_and_localization
+[I18N]: http://www.i18nguy.com/origini18n.html
+[localisation]: http://www.translationzone.com/solutions/software-localization/
+[business logic]: http://www.paragoncorporation.com/ArticleDetail.aspx?ArticleID=21
+[tools]: https://en.wikibooks.org/wiki/FOSS_Open_Standards/Standards_and_Internationalization/Localization_of_Software

publishing
diff --git a/posts/semantic-dollar-thing.mdwn b/posts/semantic-dollar-thing.mdwn
new file mode 100644
index 0000000..9578ab2
--- /dev/null
+++ b/posts/semantic-dollar-thing.mdwn
@@ -0,0 +1,46 @@
+[[!meta date="Wed, 01 Feb 2017 12:00:06 +0000"]]
+[[!meta title="Semantic $THING"]]
+[[!meta author="Daniel Silverstone"]]
+
+There has been, of late, a significant gain in mindshare being enjoyed by a
+number of movements which call themselves _Semantic $THING_. These usually
+encode some extra meaning in content which otherwise exists.  Commonly these
+are intended to make it easier for computers to read extra meaning into
+something intended for humans; but sometimes they're intended to allow humans
+to more easily deal with something meant for computers.
+
+[Semantics][] are, in a very basic sense, how meaning is overlayed onto the
+[syntax][] of content.  I like to think of it as: syntax is the 'how', but
+semantics are the 'what'.
+
+[Semantics]: https://en.wikipedia.org/wiki/Semantics
+[Syntax]: https://en.wikipedia.org/wiki/Syntax
+
+Here are three _Semantic $THINGs_ which I think you ought to know about:
+
+* The
+  [Semantic Linefeeds](http://rhodesmill.org/brandon/2012/one-sentence-per-line/)
+  concept is intended to make it easier for humans to [grok][] the [delta][]
+  between two versions of a text file intended to be processed (for example
+  [markdown][])
+
+* The [Semantic Versioning](http://semver.org/) concept is intended to make it
+  possible for humans, _and_ software, to understand the relationship between
+  different releases of a piece of software.
+
+* The
+  [Semantic Commits](https://seesparkbox.com/foundry/semantic_commit_messages)
+  concept is intended to make it easier to produce [changelogs][] for projects
+  and there's a number of tools built up around this.
+
+If you know of any other useful _Semantic $THINGs_ then why not comment on this
+article to let others know about them?  For homework, I simply suggest you read
+the above linked articles, and then do a little of your own research around the
+topics and consider if you might need to take on any of the points in your own
+projects.  I am considering semantic commits for my main projects at the time
+of writing this article.
+
+[grok]: https://en.wikipedia.org/wiki/Grok
+[delta]: https://en.wikipedia.org/wiki/Delta_encoding
+[markdown]: https://en.wikipedia.org/wiki/Markdow
+[changelogs]: http://keepachangelog.com/en/0.3.0/

creating tag page tags/standard
diff --git a/tags/standard.mdwn b/tags/standard.mdwn
new file mode 100644
index 0000000..6838980
--- /dev/null
+++ b/tags/standard.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged standard"]]
+
+[[!inline pages="tagged(standard)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/files
diff --git a/tags/files.mdwn b/tags/files.mdwn
new file mode 100644
index 0000000..0ff2fdd
--- /dev/null
+++ b/tags/files.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged files"]]
+
+[[!inline pages="tagged(files)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/xdg
diff --git a/tags/xdg.mdwn b/tags/xdg.mdwn
new file mode 100644
index 0000000..7a249a2
--- /dev/null
+++ b/tags/xdg.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged xdg"]]
+
+[[!inline pages="tagged(xdg)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/xdg-dirs.mdwn b/posts/xdg-dirs.mdwn
new file mode 100644
index 0000000..cde572e
--- /dev/null
+++ b/posts/xdg-dirs.mdwn
@@ -0,0 +1,58 @@
+[[!meta date="Wed, 25 Jan 2017 12:00:07 +0000"]]
+[[!meta title="The XDG base directory standard"]]
+[[!meta author="Richard Maw"]]
+[[!tag xdg fhs standard files]]
+
+We previously spoke of the FHS directory standard.
+
+This includes important rules for where a program should store its data.
+This is particularly important when writing your programs
+as you need to know where you should write your program state to.
+
+While this answers where you should write your state to
+for programs that run system-wide,
+it does not help for per-user programs.
+
+The traditional unix approach for this has been [dotfiles][]
+so each program leaves hidden files all over the user's home directory.
+
+This is sub-optimal
+since the directories are hidden by default,
+which solves not cluttering the file listing in your home directory
+at the cost of merely hiding the clutter rather than structuring it.
+
+To plug this gap the [XDG directory standard][] was designed.
+
+Rather than it specifying exactly which directories should be used,
+it specifies an environment variable which descibes which directory to use
+and a default if the environment variable is not defined.
+
+[[!table data="""
+Variable         |Default       |FHS equivalent|Purpose
+`XDG_CONFIG_HOME`|~/.config     |/etc          |Program configuration files.
+`XDG_CACHE_HOME` |~/.cache      |/var/cache    |Non-essential cached data.
+`XDG_DATA_HOME`  |~/.local/share|/usr/share    |Data files.
+`XDG_RUNTIME_DIR`|              |/var/run      |Program state for current boot.
+"""]]
+
+The standard also defines `XDG_DATA_DIRS` and `XDG_CONFIG_DIRS`,
+but they are about where to read system-wide config files from.
+
+The rules are meant to be simple enough that you can implement it yourself,
+but there are some helper libraries available.
+
+If you're writing python there is [PyXDG][].
+If you're writing C there's [implementations in GLib][],
+or a 3-clause BSD licensed implementation in [chck][].
+
+If you're interested in more standards for where to put files,
+a relevant [XDG user directory standard][] exists,
+which lists more variables in `${XDG_CONFIG_HOME}/user-dirs.dirs`
+for directories like the default place to download files to.
+
+[dotfiles]: https://en.wikipedia.org/wiki/Hidden_file_and_hidden_directory#Unix_and_Unix-like_environments
+[XDG directory standard]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
+[PyXDG]: https://freedesktop.org/wiki/Software/pyxdg/
+[implementations in GLib]: https://developer.gnome.org/glib/stable/glib-Miscellaneous-Utility-Functions.html#g-get-user-cache-dir
+[chck]: https://github.com/Cloudef/chck/tree/55d41fbf2c9f58500a0dca8402a3151c6e50742a/chck/xdg
+[XDG user directory standard]: https://www.freedesktop.org/wiki/Software/xdg-user-dirs/

publishing
diff --git a/posts/credit-where-credit-is-due.mdwn b/posts/credit-where-credit-is-due.mdwn
new file mode 100644
index 0000000..9643b52
--- /dev/null
+++ b/posts/credit-where-credit-is-due.mdwn
@@ -0,0 +1,24 @@
+[[!meta date="Wed, 18 Jan 2017 12:00:06 +0000"]]
+[[!meta title="Give credit where credit is due"]]
+[[!meta author="Daniel Silverstone"]]
+
+If you've been lucky enough to write some software worth publishing, and then
+super-lucky enough to have others like it enough to use it; and then ultra-mega
+lucky enough to have someone to like it enough to send you a patch for it; then
+you may be lucky enough to need to have a [credits][credits1] [file][credits2].
+
+[credits1]: https://github.com/torvalds/linux/blob/master/CREDITS
+[credits2]: https://github.com/showdownjs/showdown/blob/master/CREDITS.md
+
+When people contribute to your projects, it's only polite to make a little note
+so that they can see that their contribution was appreciated.  Some of my
+projects even [credit my employer][credits3] when I've been fortunate enough to
+be allowed to write some F/LOSS at work.
+
+[credits3]: https://git.gitano.org.uk/gitano.git/tree/CREDITS
+
+The same applies if you "borrow" code from another project.  Be sure to comply
+with their licensing terms, and even if they don't require it, it's only polite
+to thank them for their work.  So for your homework, I'd like you to go back to
+any software you've written which has received patches from others and be sure
+to credit your contributors clearly and fully.

creating tag page tags/fhs
diff --git a/tags/fhs.mdwn b/tags/fhs.mdwn
new file mode 100644
index 0000000..ebd55eb
--- /dev/null
+++ b/tags/fhs.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged fhs"]]
+
+[[!inline pages="tagged(fhs)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/hierarchy
diff --git a/tags/hierarchy.mdwn b/tags/hierarchy.mdwn
new file mode 100644
index 0000000..a7a2237
--- /dev/null
+++ b/tags/hierarchy.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged hierarchy"]]
+
+[[!inline pages="tagged(hierarchy)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/fhs.mdwn b/posts/fhs.mdwn
new file mode 100644
index 0000000..7eb40be
--- /dev/null
+++ b/posts/fhs.mdwn
@@ -0,0 +1,75 @@
+[[!meta date="Wed, 11 Jan 2017 12:00:06 +0000"]]
+[[!meta title="FHS, the filesystem hierarchy standard"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag fhs hierarchy]]
+
+Unix systems have a number of traditions and conventions for what goes
+where in the filesystem. Linux follows Unix.
+
+A Unix system has directories such as `/bin` and `/usr/bin` for
+executables, and the difference may be a bit mysterious. It gets
+worse, though, once you realise there is at least **six** other
+directories for executables: `/sbin`, `/usr/sbin`, `/usr/local/bin`,
+`/usr/local/sbin`.
+
+In addition to its binaries, a program may consist of data files,
+shared libraries, and documention, and it may need to write various
+files for different purposes. The Unix tradition is to spread these
+into different areas of the filesystem tree.
+
+Some other operating systems have a different approach, often one
+where all the parts of a program are put in the same place. Unix
+instead combines like with like: executables in one place (or six),
+documentation in another, and data files in a third. A common
+justification is that, say, backing up only the data files becomes
+easier: everything else can just be reinstalled, if need be.
+
+Early on in the history of Linux, there were big differences between
+where different Linux distributions put files. A consensus grew that a
+standard would be beneficial, and so FSSTND, the Linux file system
+standard, was born. Some years later, its scope was broadened to all
+free software implementations of Unix and it got renamed to [FHS][].
+
+[FHS]: https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
+[standard]: http://refspecs.linuxfoundation.org/fhs.shtml
+
+The FHS Wikipedia page gives an overview. The actual [standard][]
+is quite readable and is recommended for when you have time.
+Additionally the [hier][](7) also gives an overview. We refer you to
+those rather than writing our own summary
+
+[hier]: https://linux.die.net/man/7/hier
+
+Things in the filesystem standards world change. The directory tree is
+alive. For example:
+
+* `/var/run` (runtime information, cleared on boot) is moving to
+  `/run` since keeping it in `/var` doesn't make anything better.
+
+* Historically the Unix directory tree is split across two or more
+  disks. The root disks has `/etc`, and `/bin` and more. A second
+  disk is mounted as `/usr` and contains `/usr/bin` among other
+  things. This is getting changed so that the directories with static
+  data on the root disk get moved to `/usr` and replaced with symlinks
+  (e.g., `/bin -> /usr/bin`). After that, the root disk only has a
+  minimal amount of variable data, such as `/etc`. Thus, almost the
+  entire operating system is on a read-only disk, which has benefits
+  for reliability, and can allow sharing it between, say, virtual
+  machines or containers.
+
+  This seems to a remarkably controversial change. Change is scary.
+
+How did the Unix directory tree grow into the sprawling [ent][] that it is
+today? It happened mostly through incremental change, not by design.
+You could almost say it happened organically.
+
+[ent]: https://en.wikipedia.org/wiki/Ent
+
+For example, the `/usr` directory was originally, in the early 1970s,
+meant for home directories. "Usr" is short for "user" in various
+parts of Unix. What happened was the the Unix developers had two disks
+in their machine, one for the system, the other for user home
+directories. They ran out of disk space on the system disk, and fixed
+this by moving parts of the system to the user disk, eventually
+crowding out users from `/usr` to `/home`. As a result we now have the
+system spread on the "root disk", and the "user disk" (`/usr`).

Fix missing braces in example;
diff --git a/posts/moving-files-2-sparseness/my-mv.c b/posts/moving-files-2-sparseness/my-mv.c
index bedd1c2..0b0edc5 100644
--- a/posts/moving-files-2-sparseness/my-mv.c
+++ b/posts/moving-files-2-sparseness/my-mv.c
@@ -45,9 +45,10 @@ ssize_t copy_range(int srcfd, int tgtfd, size_t range) {
 
         while (n_read > 0) {
             ssize_t n_written = TEMP_FAILURE_RETRY(write(tgtfd, buf, n_read));
-            if (n_written < 0)
+            if (n_written < 0) {
                 perror("Write to target file");
                 return n_written;
+            }
 
             n_read -= n_written;
             copied += n_written;

publishing
diff --git a/posts/writing-for-others.mdwn b/posts/writing-for-others.mdwn
new file mode 100644
index 0000000..da2fecd
--- /dev/null
+++ b/posts/writing-for-others.mdwn
@@ -0,0 +1,48 @@
+[[!meta date="Wed, 04 Jan 2017 12:00:08 +0000"]]
+[[!meta title="Writing for others"]]
+[[!meta author="Daniel Silverstone"]]
+
+We've often spoken about how it's good to write software which others will want
+to use.  This is really important but it's only one of the various undertakings
+which will transform you from a hacker into a good software author.  What your
+users want the software to do is only one aspect of their needs when it comes
+to any programming you do.  In addition there's the aspect of *where* they want
+it to work.  Now, I'm not going to suggest that you must make your software work
+everywhere.  But I am going to suggest that you need to consider where your
+users live in terms of their software environment.
+
+If you're having to use lots of bleeding edge libraries in your software
+development then your users will need to have those libraries in order to use
+your software.  If you're targetting people who use [Arch][] or similar then
+that might be just fine, but if you're hoping that people who use the stable
+version of [Debian][] or [Fedora][] then you might need to consider using less
+bleeding-edge dependencies.
+
+Another solution could be to include the libraries you depend on inside your
+software, but that tends to make it significantly harder to manage your
+software as part of a distribution.  Or you might make your build system
+download the new libraries as part of compiling/installing your software in the
+hope that will help.  Unfortunately many build systems are deliberately not
+allowed to talk to the Internet to improve reproducibility which will scupper
+your plans.
+
+Some languages have their own packaging solution which can be used to
+automatically deal with this to a greater or lesser extent, but they have the
+side-effect of constructing confusing messes on your users' systems unless they
+are proficient with ways to construct virtual environments for whatever
+language you have selected.
+
+Instead, the best option for increasing the usability of your software is,
+simply, to ensure that you don't depend on things which are not already in the
+distributions you are targetting.  This doesn't mean that you *must* limit
+yourself to the stable version of a distribution; particularly if you're
+targetting your software at the distribution's next release; but if you can
+limit yourself thusly then it will be must easier to have a wide user base.
+This option also has the benefit that if there's a security issue in any of
+your dependencies, the distribution will take care of that for you, reducing
+one potential source of headaches for you.
+
+[Arch]: https://www.archlinux.org/
+[Debian]: https://www.debian.org/
+[Fedora]: https://getfedora.org/
+

Added a comment: re: thoughts on naive sparse copy
diff --git a/posts/moving-files-2-sparseness/comment_2_7a4102bc9acc4ca9abc2d7b7a9434f9b._comment b/posts/moving-files-2-sparseness/comment_2_7a4102bc9acc4ca9abc2d7b7a9434f9b._comment
new file mode 100644
index 0000000..687d928
--- /dev/null
+++ b/posts/moving-files-2-sparseness/comment_2_7a4102bc9acc4ca9abc2d7b7a9434f9b._comment
@@ -0,0 +1,26 @@
+[[!comment format=mdwn
+ username="http://richard.maw.name/"
+ nickname="Richard Maw"
+ avatar="http://cdn.libravatar.org/avatar/4d312a0989b085a9dfa9ba61d60da130979b9c408659132fea11d5efe2c90fc5"
+ subject="re: thoughts on naive sparse copy"
+ date="2017-01-03T13:50:11Z"
+ content="""
+> thanks for another massively useful article. :)
+>  
+> I didn't know about SEEK_HOLE and SEEK_DATA at all. As far as I was aware the traditional way to do a sparse copy is set some bound on the number of zeroes that will constitute a hole and when n or more such zeroes are read, instead of copying the zeroes we seek past the current destination offset by n. This has the obvious disadvantage that any holes present in the source file that are smaller than n bytes will not be preserved as holes in the destination.
+
+There's also the disadvantage of taking a lot longer to read since you have to copy those zeroes into userspace and scan through them to see if they are indeed all zeroes.
+I think this is what `tar`, `rsync` and `cp --sparse=always` does, but I'd have to check.
+
+It's also not fun for disk images, beyond the size difference, since you want to avoid writing what you don't need to when writing a disk image to a real disk, but you also need to write zeroes that are in the disk image, since you can end up with an invalid file system if you don't write those zeroes.
+https://source.tizen.org/documentation/reference/bmaptool/introduction was written by the Tizen guys for dealing with this. It produces and uses a separate file for the map of which sectors to copy, possibly because I know of no serialisation formats other than a disk image that preserves holes without dropping zero data sectors.
+
+> My man page for lseek(2) says that SEEK_HOLE and SEEK_DATA are only supported for more recent linux kernels and even then just for a selection of file systems: btrfs (3.1), OCFS (3.2), XFS (3.5), ext4 (3.8) and tmpfs (3.8). I wonder then whether it's worthwhile making a sparse copy function that falls back to the traditional sparse copy method if the new way is unsupported? (or maybe I'm living in the past having only made the switch from ext3 in the past year or so :D )
+
+As far as I'm concerned those kernel versions are old enough that I can't feasibly test them, so I can't be sure my fallback would work right.
+I would accept a patch if provided one.
+
+There are a couple of alternative fallbacks. The `FIEMAP` ioctl can be used to get information about which parts of the file contain what types of data, and the `FIBMAP` ioctl can tell you where on disk the data is.
+`FIBMAP` is old, so more widely available, but of limited use since it requires elevated privileges.
+`FIEMAP` is older than `SEEK_{DATA,HOLE}` and newer than `FIBMAP`, but has been historically the cause of file corruption. I don't recall whether old versions were just used improperly or whether there were bugs with it, but I steered clear of it and opted for the simpler interface of `SEEK_{HOLE,DATA}`.
+"""]]

Added a comment: re: thanks/eintr
diff --git a/posts/moving-files-1-copying/comment_2_9dd913733a1ab20cb1415eae8eac924f._comment b/posts/moving-files-1-copying/comment_2_9dd913733a1ab20cb1415eae8eac924f._comment
new file mode 100644
index 0000000..da0d538
--- /dev/null
+++ b/posts/moving-files-1-copying/comment_2_9dd913733a1ab20cb1415eae8eac924f._comment
@@ -0,0 +1,28 @@
+[[!comment format=mdwn
+ username="http://richard.maw.name/"
+ nickname="Richard Maw"
+ avatar="http://cdn.libravatar.org/avatar/4d312a0989b085a9dfa9ba61d60da130979b9c408659132fea11d5efe2c90fc5"
+ subject="re: thanks/eintr"
+ date="2017-01-03T13:20:58Z"
+ content="""
+>   Thanks for this article it's really useful, it would be interesting to know why you're not fond of the stdio interface,
+
+I don't like the buffering behaviour, it defers writes late enough that I lose context about what bit of the write failed, so when I go to flush and close I can't say how much was actually written.
+
+>   and also potentially worth mentioning that EINTR is really only something that needs to be explicitly handled on Linux, from signal(7):
+>
+>     On Linux, even in the absence of signal handlers, certain blocking interfaces  can
+>     fail  with the error EINTR after the process is stopped by one of the stop signals
+>     and then resumed via SIGCONT.  This behavior is not  sanctioned  by  POSIX.1,  and
+>     doesn't occur on other systems.
+>
+>   This behaviour seems less than helpful to me, it would be really good to know if there's a good reason why GNU/Linux doesn't just restart the call (as the BSDs do)
+
+There's also the fact that some signals can be configured to restart, but not others.
+Signals in general are a bit of a mess, so my way of dealing with it is to wrap everything in a retry and leave signal handling for shut down and use a better form of IPC for everything else.
+
+>   Also, I had no idea you could make system calls without having a C wrapper for them, so thanks for that as well!
+
+Yep, it's a pain when the wrappers don't exist because you need to handle the error return calling convention yourself, since part of what the libc wrappers do is set errno.
+I tend to use negative error number returns in my own code rather than errno which makes it actually closer to what I prefer.
+"""]]

Added a comment: thoughts on naive sparse copy
diff --git a/posts/moving-files-2-sparseness/comment_1_14de170e2381447be1468798aa444897._comment b/posts/moving-files-2-sparseness/comment_1_14de170e2381447be1468798aa444897._comment
new file mode 100644
index 0000000..2eb7f84
--- /dev/null
+++ b/posts/moving-files-2-sparseness/comment_1_14de170e2381447be1468798aa444897._comment
@@ -0,0 +1,23 @@
+[[!comment format=mdwn
+ username="Gravious"
+ avatar="http://cdn.libravatar.org/avatar/48cd8810cb554be5891de36dd031b49c"
+ subject="thoughts on naive sparse copy"
+ date="2017-01-01T14:33:17Z"
+ content="""
+thanks for another massively useful article. :)
+
+I didn't know about SEEK_HOLE and SEEK_DATA at all.
+As far as I was aware the traditional way to do a sparse copy is
+set some bound on the number of zeroes that will constitute a hole
+and when n or more such zeroes are read, instead of copying the zeroes
+we seek past the current destination offset by n. This has the obvious disadvantage
+that any holes present in the source file that are smaller than n bytes
+will not be preserved as holes in the destination.
+
+My man page for lseek(2) says that SEEK_HOLE and SEEK_DATA are only
+supported for more recent linux kernels and even then just for a selection of file systems:
+btrfs (3.1), OCFS (3.2), XFS (3.5), ext4 (3.8) and tmpfs (3.8).
+I wonder then whether it's worthwhile making a sparse copy function that
+falls back to the traditional sparse copy method if the new way is unsupported?
+(or maybe I'm living in the past having only made the switch from ext3 in the past year or so :D )
+"""]]

calendar update
diff --git a/archives/2017.mdwn b/archives/2017.mdwn
new file mode 100644
index 0000000..592399b
--- /dev/null
+++ b/archives/2017.mdwn
@@ -0,0 +1 @@
+[[!calendar type=year year=2017 pages="page(posts/*) and !*/Discussion"]]
diff --git a/archives/2017/01.mdwn b/archives/2017/01.mdwn
new file mode 100644
index 0000000..2441b7a
--- /dev/null
+++ b/archives/2017/01.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=01 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(01) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/02.mdwn b/archives/2017/02.mdwn
new file mode 100644
index 0000000..e295fbe
--- /dev/null
+++ b/archives/2017/02.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=02 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(02) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/03.mdwn b/archives/2017/03.mdwn
new file mode 100644
index 0000000..6c05242
--- /dev/null
+++ b/archives/2017/03.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=03 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(03) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/04.mdwn b/archives/2017/04.mdwn
new file mode 100644
index 0000000..76e7c08
--- /dev/null
+++ b/archives/2017/04.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=04 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(04) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/05.mdwn b/archives/2017/05.mdwn
new file mode 100644
index 0000000..678f63a
--- /dev/null
+++ b/archives/2017/05.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=05 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(05) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/06.mdwn b/archives/2017/06.mdwn
new file mode 100644
index 0000000..2887ab6
--- /dev/null
+++ b/archives/2017/06.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=06 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(06) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/07.mdwn b/archives/2017/07.mdwn
new file mode 100644
index 0000000..0f746b9
--- /dev/null
+++ b/archives/2017/07.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=07 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(07) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/08.mdwn b/archives/2017/08.mdwn
new file mode 100644
index 0000000..4da2722
--- /dev/null
+++ b/archives/2017/08.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=08 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(08) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/09.mdwn b/archives/2017/09.mdwn
new file mode 100644
index 0000000..70f5e1d
--- /dev/null
+++ b/archives/2017/09.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=09 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(09) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/10.mdwn b/archives/2017/10.mdwn
new file mode 100644
index 0000000..04f5435
--- /dev/null
+++ b/archives/2017/10.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=10 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(10) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/11.mdwn b/archives/2017/11.mdwn
new file mode 100644
index 0000000..fe53f8d
--- /dev/null
+++ b/archives/2017/11.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=11 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(11) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]
diff --git a/archives/2017/12.mdwn b/archives/2017/12.mdwn
new file mode 100644
index 0000000..bb4a435
--- /dev/null
+++ b/archives/2017/12.mdwn
@@ -0,0 +1,5 @@
+[[!sidebar content="""
+[[!calendar type=month month=12 year=2017 pages="page(posts/*) and !*/Discussion"]]
+"""]]
+
+[[!inline pages="creation_month(12) and creation_year(2017) and page(posts/*) and !*/Discussion" show=0 feeds=no reverse=yes]]

Added a comment: thanks/eintr
diff --git a/posts/moving-files-1-copying/comment_1_89abf821373c4e306619a2b81ab80140._comment b/posts/moving-files-1-copying/comment_1_89abf821373c4e306619a2b81ab80140._comment
new file mode 100644
index 0000000..d7f5157
--- /dev/null
+++ b/posts/moving-files-1-copying/comment_1_89abf821373c4e306619a2b81ab80140._comment
@@ -0,0 +1,22 @@
+[[!comment format=mdwn
+ username="Gravious"
+ avatar="http://cdn.libravatar.org/avatar/48cd8810cb554be5891de36dd031b49c"
+ subject="thanks/eintr"
+ date="2016-12-30T20:47:34Z"
+ content="""
+Thanks for this article it's really useful,
+it would be interesting to know why you're not fond of the stdio interface,
+and also potentially worth mentioning that EINTR is really only something
+that needs to be explicitly handled on Linux, from signal(7):
+
+       On Linux, even in the absence of signal handlers, certain blocking interfaces  can
+       fail  with the error EINTR after the process is stopped by one of the stop signals
+       and then resumed via SIGCONT.  This behavior is not  sanctioned  by  POSIX.1,  and
+       doesn't occur on other systems.
+
+This behaviour seems less than helpful to me,
+it would be really good to know if there's a good reason why GNU/Linux doesn't just restart the call (as the BSDs do)
+
+Also, I had no idea you could make system calls without having a C wrapper for them,
+so thanks for that as well!
+"""]]

publishing
diff --git a/posts/tidy-your-code.mdwn b/posts/tidy-your-code.mdwn
new file mode 100644
index 0000000..bc554a9
--- /dev/null
+++ b/posts/tidy-your-code.mdwn
@@ -0,0 +1,122 @@
+[[!meta date="Wed, 28 Dec 2016 12:00:06 +0000"]]
+[[!meta title="If you don't tidy your code, you won't be getting any contributors this week!"]]
+[[!meta author="Richard Maw"]]
+[[!tag C open-source]]
+
+This is of course a pun on the old phrase
+"If you don't tidy your room, you won't be getting any pocket money this week."
+in which privileges are withdrawn if tedious chores aren't completed.
+
+It's not always fun to do the cleanup
+that follows in the wake of feature development.
+
+Like the admonishment towards children,
+this has motivations beyond immediate gratification.
+If you don't tidy your codebase, it will begin to [smell][code smell].
+This smell is your [technical debt][] building up.
+
+If you don't clean it up,
+it will become harder to do so in the future
+and it will also be more difficult to implement features.
+
+Unlike most people's bedrooms however,
+you _want_ to let everyone into your codebase
+so that other people can enjoy it and improve it.
+
+Having an untidy codebase makes this difficult,
+because it's requires a significant incentive to work on an untidy codebase.
+Why would anyone else want to tidy your mess?
+
+It's also worth noting that you from a month ago is a different person,
+so some projects can die from the original developer coming back
+to the mess they left and deciding it wasn't worth tidying.
+
+So if you have a project that you have not touched in a while
+try to make some time to tidy up some of the mess.
+
+It helps if you have left a `TODO` file to tell you what needs to be tidied
+rather than having to read through your codebase again to find the smells.
+
+# Tips for tidying C projects
+
+It would be hypocritical for me to make these sweeping generalisations
+if I then did not follow through with it,
+so I am yet again going to use my [linux-fsops][] project as the example.
+
+1.  Tidy your source files into a directory of their own.
+
+    Your project is more than just your code.
+    There will be other artifacts you want to version control
+    and at least in the short term one of the best places for this
+    is your source code repository.
+
+    So sorting your artifacts into a nice hierarchical tree helps
+    especially when you need to keep track of things.
+
+    It is also valuable in some programming environments.
+    For example, Rust expects to have a `src` directory if you are using cargo,
+    and for python projects it is handy to separate your code out
+    into a library named after your project
+    since this allows you to automate package generation.
+
+2.  Split up large source files.
+
+    Many helper functions would be usefully shared within your project
+    and if you've defined your functions with low coupling
+    then they may be useful independently,
+    so splitting those functions out into separate source files
+    is a step towards having a library that would be useful to others.
+
+    Once every usefully divisible function has been split out,
+    mark the rest as `static`, partly to ensure their defenitions don't leak,
+    but also to signal intent that they aren't shared.
+
+    Later steps may include turning it into an actual library
+    by changing it to build as a [shared object][],
+    having it install headers separately,
+    and shipping the headers and [shared object][] as separate packages,
+    but until then it is still useful as a copy-library.
+
+    Of special note is to have a `missing.h` for all the definiions
+    that are not available from your system headers.
+    It is conveneient to split it out this way
+    since if you later decide to move to [automake][]
+    it has support for generating it automatically.
+
+3.  Tidy your build-system.
+
+    This is often the most-overlooked part of a project
+    as it doesn't make the result itself better,
+    but it makes it easier to make those improvements.
+
+    In a Makefile it is necessary to ensure your make rules'
+    dependencies and clean rules are correct.
+
+4.  Don't worry about tidying everything at once.
+
+    While it's better to have a completely tidied codebase,
+    requiring that as a goal is likely to kill any enthusiasm in doing so.
+
+    Incremental improvements are great.
+    Every step is a step towards the goal.
+    Perfect is the enemy of good.
+    Other platitudes are available.
+
+    Don't get hung up on the fact that your codebase still has stuff to tidy.
+    So long as you've done something to improve it,
+    you have not wasted your time.
+    You can always do more later.
+
+To to this end I have tidied up the build-system
+and [split funcionality out in my `linux-fsops` project](fsops-commit).
+
+Now, if you're feeling particularly generous,
+you might think of a project you are involved in,
+and give the gift of tidy code this festive period!
+
+[code smell]: https://en.wikipedia.org/wiki/Code_smell
+[technical debt]: https://en.wikipedia.org/wiki/Technical_debt
+[linux-fsops]: https://github.com/fishface60/linux-fsops
+[automake]: https://www.gnu.org/software/automake/
+[shared object]: https://en.wikipedia.org/wiki/Library_(computing)#Shared_libraries
+[fsops-commit]: https://github.com/fishface60/linux-fsops/commit/dcd12642bb605d06b71ae0ef57acaea5a7f379e2

publishing
diff --git a/posts/giving-presentations.mdwn b/posts/giving-presentations.mdwn
new file mode 100644
index 0000000..139c35f
--- /dev/null
+++ b/posts/giving-presentations.mdwn
@@ -0,0 +1,83 @@
+[[!meta date="Wed, 21 Dec 2016 12:00:07 +0000"]]
+[[!meta title="Giving Presentations"]]
+[[!meta author="Daniel Silverstone"]]
+
+Some people give gifts at this time of year, I prefer to give presentations.
+Well, I like doing it at any time of year really; but I thought I ought to try
+and make some concession to the time of year.
+
+At most conferences I go to, I end up giving a presentation (and no, that's not
+just because you can sometimes get paid to attend if you speak) and so I've
+tried most of the Linux-based presentation tools over the years.  I'm sure
+there are new ones I haven't yet encountered, but I thought I might include
+some of the ones I've been at least marginally successful using.  If you are
+wanting to give presentations from a Linux laptop then you might find the
+following useful.
+
+Things I've used
+================
+
+Libreoffice Impress
+-------------------
+
+If you're used to "_traditional_" "_powerpoint_" "_presentations_" then you
+will likely find [Libreoffice][]'s presentation tool to be very easy to get
+used to.  It is quite easy to get basic presentations going with relative ease.
+It is not, however, very modern in its presentation design, which can result in
+quite dated-looking slides.
+
+Pinpoint
+--------
+
+If you prefer a simpler looking presentation style then you might enjoy the
+[Pinpoint][] from the Gnome project.  Pinpoint is a very simple presentation
+tool which focusses on providing simple slides which look clean, and making it
+nice and easy to manage your presenter notes.  Unlike Libreoffice, Pinpoint's
+input is a plain text file which makes it much easier to version your
+presentation in Git.
+
+PDF Presenter Console
+---------------------
+
+If, on the other hand, you prefer to generate your presentation in some other
+tool which can then output a PDF; then [pdf-presenter-console][] may be for
+you.  It takes a PDF (and optionally some presenter notes) and provides a
+multi-screen presentation tool with presenter notes.  You can edit your notes
+in the tool if you notice problems during your test runs, and it's a good
+example of doing one thing and doing it well.
+
+Things I've not used
+====================
+
+I know various people who use [Pandoc][] combined with [slidy][] or even write
+their own [LaTeX][] with [beamer][] or similar.  Personally I think that if I
+used those, I'd then want to use `pdf-presenter-console` to give me speaker
+notes capability though a PDF with no notes is a useful way to have a
+presentation which is easy to transfer to other computers.
+
+As an alternative to slides, some people prefer to type messages on the fly. To
+do this you can use [screen-message][] which displays text as large as possible
+as it is typed.  This is useful for when you improvise a presentation (or want
+to make it appear improvised).
+
+A few tips
+==========
+
+If you're new to presentations then it can be very tempting to fill your slides
+with useful information so that you feel like your audience will get a lot out
+of your talk.  This is a _bad thing_™ and you shouldn't do it.  In fact there's
+so much useful out there about how to give presentations that I'd suggest you
+go and look at things like
+[this](http://www.skillsyouneed.com/present/presentation-tips.html).
+
+
+
+
+[Libreoffice]: https://www.libreoffice.org/
+[Pinpoint]: https://wiki.gnome.org/action/show/Apps/Pinpoint
+[pdf-presenter-console]: https://pdfpc.github.io/
+[Pandoc]: http://pandoc.org/
+[slidey]: https://www.w3.org/Talks/Tools/Slidy2/Overview.html#(1)
+[LaTeX]: https://www.latex-project.org/
+[beamer]: https://www.sharelatex.com/learn/Beamer
+[screen-message]: https://www.joachim-breitner.de/projects#screen-message

creating tag page tags/fluff
diff --git a/tags/fluff.mdwn b/tags/fluff.mdwn
new file mode 100644
index 0000000..2403361
--- /dev/null
+++ b/tags/fluff.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged fluff"]]
+
+[[!inline pages="tagged(fluff)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/inspirational
diff --git a/tags/inspirational.mdwn b/tags/inspirational.mdwn
new file mode 100644
index 0000000..0c4ba53
--- /dev/null
+++ b/tags/inspirational.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged inspirational"]]
+
+[[!inline pages="tagged(inspirational)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/inspiration.mdwn b/posts/inspiration.mdwn
new file mode 100644
index 0000000..66952ef
--- /dev/null
+++ b/posts/inspiration.mdwn
@@ -0,0 +1,48 @@
+[[!meta date="Wed, 14 Dec 2016 12:00:07 +0000"]]
+[[!meta title="Achieving simplicity and greatness"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag fluff inspirational]]
+
+Something that repeatedly comes up about free software projects is
+how much it's possible to achieve. Can you write an operating system
+from scratch? That's been done, several times. A desktop environment?
+Been done. A complete toolchain (compiler, linker, essential
+libraries)? Yeah, done.
+
+These are all examples of large, impressive systems that have been
+made by one or a small number of people, even just in their free time.
+How is that possible? Here's a couple of ideas.
+
+* Have fun. If you're not enjoying it, and you're not getting paid
+  (and maybe even then), you should perhaps do something else. Or
+  something differently. Is it not fun because memory management in C
+  is tedious? Maybe use a different language or some helper libraries
+  that take the tediousness away. Is it not challenging enough? Make N
+  bigger.
+
+* Do not believe the people who say your idea isn't plausible to do.
+  They might be right, and if you idea is ambitious, it might take a
+  long time to get it done. Believing everyone who says you can't is
+  just demoralising, especially online where there are always at least
+  12765 people awake and happy to denigrate anyone else's ideas.
+
+  Do it. If you fail, you've had fun for a while and you probably
+  learnt a thing or two. If you don't even try, those who don't
+  believe in you were right.
+
+* Start small and simple, then grow organically. No, smaller and
+  simpler than that. There is much truth in Antoine de Saint Exupéry's
+  saying, "It seems that perfection is attained not when there is
+  nothing more to add, but when there is nothing more to remove.".
+  Your first version should be as small and simple as you can make it.
+  If you can think of anything to remove that doesn't render the thing
+  unusable for all use cases, you can simplify further.
+
+  When you have the smallest, simplest first version that is possible,
+  you can start growing. Get a user (even if it is just you), and
+  whenever they want a new feature, and that feature fits in your
+  vision of your program, implement that next. Repeat a thousand
+  times. Keep doing it. Find others to help. Don't give up. Persevere.
+  It might take a year or ten, but you'll eventually have something
+  that makes people wonder how you managed to make something so
+  impressive.

publishing
diff --git a/posts/code-for-fun.mdwn b/posts/code-for-fun.mdwn
new file mode 100644
index 0000000..9d408c8
--- /dev/null
+++ b/posts/code-for-fun.mdwn
@@ -0,0 +1,46 @@
+[[!meta date="Wed, 07 Dec 2016 12:00:07 +0000"]]
+[[!meta title="Code for fun"]]
+[[!meta author="Daniel Silverstone"]]
+
+Plenty of times now, we've exhorted you to write software which will be useful
+for others.  We've also asked you to ensure that you know what you're up to and
+that you've planned well in advance.  But once or twice we've mentioned that
+doing something "for fun" is plenty reason enough.
+
+When learning a new programming language, I (and others that I know) have a
+simple ritual set of small programs that I implement in order to get to know
+how a language works.  For example, I try and implement simple problems from
+the [Project Euler][] website, along with some simple unixy commands such as
+a basic `cat`, `echo`, `ls` set.   A friend of mine always tries to implement
+a simple number guessing game because she likes interactive programs.
+
+Learning a new programming language, especially one which isn't quite like the
+ones you use day-to-day can be a very humbling experience; especially if you
+don't let yourself start by modifying someone else's code.  It can help to
+remind us that we're all levelled by something which isn't quite in our realm
+of normal thought.  At this time of year I like to learn a new language to see
+if the following year I should have a go at something new.  Of the various
+resources out there with programming puzzles, something new started last year
+and has continued this year - [The Advent of Code][].  Last year I did it in
+[Haskell][] because I wasn't quite ready to learn something completely new, but
+did want to improve my confidence in Haskell.  This year I'm trying out
+[Rust][].
+
+We're only a week into [The Advent of Code][] and that's not quite enough to
+make it impossible to catch up.  While I'm not suggesting you **must** do it in
+a different language from those you are normally comfortable in; I am
+suggesting that to work your way through a set of programming puzzles can be
+extremely satisfying, and I encourage you all to have a go.  If doing so at
+this time of year isn't quite for you, you can always revisit the puzzles in
+the new year.
+
+And if doing something so blatantly Christmas themed isn't your thing, then I
+suggest visiting [Project Euler][] if you're mathematical, or perhaps picking
+something from the list
+[here](http://sixrevisions.com/resources/10-puzzle-websites-to-sharpen-your-programming-skills/)
+instead.
+
+[Project Euler]: https://projecteuler.net/
+[The Advent of Code]: https://adventofcode.com/
+[Haskell]: https://www.haskell.org/
+[Rust]: https://www.rust-lang.org/

publishing
diff --git a/posts/get-you-code-out-there.mdwn b/posts/get-you-code-out-there.mdwn
new file mode 100644
index 0000000..586e88d
--- /dev/null
+++ b/posts/get-you-code-out-there.mdwn
@@ -0,0 +1,304 @@
+[[!meta date="Wed, 30 Nov 2016 12:00:09 +0000"]]
+[[!meta title="Get your code out there"]]
+[[!meta author="Richard Maw"]]
+
+Code that is unused has little value.
+=====================================
+
+If nobody is using your code then how do you know if it is any good?
+
+Presumably you wrote it for a reason.
+
+Don't hide your code away, let it be free!
+
+Collaborative projects are more successful than solo projects.
+==============================================================
+
+Having others use your code and provide feedback keeps the project alive.
+
+If others take an interest and provide patches - that is excellent,
+but having people tell you that they are using your code
+is excellent motivation for you to scratch some itches yourself.
+
+Make your code easy to use for your target audience.
+====================================================
+
+I don't know the audience for my code,
+I hope to find out by its reception.
+
+If it turns out I don't have one, then so be it.
+In this case the code itself isn't the goal,
+it exists as a vehicle for writing these articles.
+
+Pick a licence
+--------------
+
+No matter how you want to make your code available
+you will need to provide a licence so anyone can be provably allowed to use it.
+
+My code is currently licensed under the [CC BY-SA 3.0][] licence,
+since that is the licence of all of the blog's contents.
+
+### Copyleft
+
+If I were to use a copyleft licence then I would be more likely
+to attract contributions from people who do not want it to be usable
+by proprietary software developers who could take it
+and not contribute any changes back.
+
+This potentially provides an incentive for potential users
+to contribute any potential improvements back.
+
+The current code is already copyleft since it's [CC BY-SA 3.0][],
+but it's less suitable as a licence for code.
+
+### Permissive
+
+A permissive licence would allow proprietary software developers
+to use my code in closed software projects.
+
+This is theoretically a wider audience than those who can use copyleft,
+but some developers may, with good reason, object to contributing to a project
+which allows others to use it without contributing anything back.
+
+It also relies on the generosity of developers to contribute any changes back
+since with a permissive licence there is no licence-provided incentive to do so.
+
+### My choice
+
+I sometimes need to write proprietary software.
+I'd like to be able to have a high quality library to use
+rather than have to re-implement it all again.
+
+I'd also prefer that people writing proprietary software use high quality code
+than implement it themselves and get it wrong.
+
+For this reason I'm going to use the ISC licence
+
+Let potential users know what licence the code is under
+-------------------------------------------------------
+
+Put a `LICENSE` file in your repository
+to make it clear how the project is licensed
+so they can know quickly whether they can use your project.
+
+Add the short text of the licence to the header of your source files
+to make sure there is no doubt that the whole of your project
+is licenced the same way.
+
+Packaging
+---------
+
+The absolute minimum is to have the source code repository publically visible,
+so anyone who wants a copy can fetch it themselves.
+
+If you then make tags it becomes easier to get a specific version.
+
+If your source code repository has a web UI it can provide snapshots,
+so users don't need to be proficient with your version control tool
+and can avoid downloading all the history they don't care about.
+
+You can also make source distributions, which may be different from a snapshot.
+These usually involve packaging up any generated or derived source files
+to reduce the number of dependencies required to build the code.
+
+If you have a set of expected target environments
+then you can provide binary packages for those environments.
+This can be a set of statically linked or bundled binaries
+designed to run on every platform.
+It could be uploaded to the platform's package repository.
+It could be participating in a distribution's community and packaging in that.
+
+Since my project is too young to even have its first release
+I'm going to just make the code available.
+As-is it is compatibly licensed such that users can copy it into their codebase.
+
+For the first release I intend to tidy it up
+into a library that can be installed by running `make install`.
+
+Put it somewhere it can be found.
+=================================
+
+Traditionally this was done by putting it in a software distribution,
+either yourself or having a member of the community package it.
+
+These days it's more common to add it to the language's package manager,
+so [CPAN][] for Perl modules, [PyPI][] for Python modules,
+[Ruby Gems][] for Ruby packages, [NPM][] for Node.js packages
+and [Crates.io][] for Rust packages.
+
+There is strong feeling about whether this is a good idea,
+but they do provide a degree of discoverability that I would like.
+
+Unfortunately there is no equivalent for C code,
+so I'm going to [throw it up on GitHub][] and hope for the best.
+
+GitHub has its faults,
+the largest of which is that it is centralising a decentralised version control,
+but like it or not, this is how much free-software is done
+and in the absence of knowing my audience I'm going to take every advantage.
+
+There's more to a project than a code dump
+==========================================
+
+Now the code is out there. Well done.
+
+But a project is more than just code.
+
+Communication
+-------------
+
+You need to communicate.
+
+When your project gets big enough you will need more than GitHub
+providing an issue tracker and an interface for merging patches.
+
+You might need an IRC channel (or other chat protocol) for providing help
+and coordinating with other developers.
+
+You might need a mailing list to receive patches,
+discuss important changes, and announce new releases.
+
+Meeting other developers and users face-to-face is important,
+so if you get big enough you might need to arrange hack days,
+dev-rooms at conferences, or eventually maybe even your own conference.
+
+It may be advisable to have a code of conduct for your community sooner
+rather than later,
+as it may help to have a statement that you wish to handle abuse seriously
+and it may prevent difficulty retrofitting a code of conduct in later.
+
+Documentation
+-------------
+
+Code documentation for how something works is handy for developers,
+more so if there are other developers who won't have your memory.
+
+Documentation of interfaces is also important,
+both internal interfaces for how implementation details interact
+and external interfaces for how your users are meant to use it,
+be it API documentation for libraries,
+usage text and man pages for command-line applications,
+or screenshotted guides for GUI applications.
+
+Then you may need more in-depth guides for how your code fits in
+to the wider context,
+such as blog articles describing how to integrate your library
+into a wider framework to achieve a particular goal,
+use your command-line application in a pipeline or batch script,
+or an example where your GUI application is used.
+
+Then there's wider-project documentation, such as plans,
+documentation of how to configure your code,
+how to set up a developemnt environment,
+where all the development resources are, etc.

(Diff truncated)
Added a comment: Re: Bugs in open_tmpfile()
diff --git a/posts/moving-files-8-atomicity/comment_2_381192724912dd814a2f5ce7daba74b1._comment b/posts/moving-files-8-atomicity/comment_2_381192724912dd814a2f5ce7daba74b1._comment
new file mode 100644
index 0000000..de1728b
--- /dev/null
+++ b/posts/moving-files-8-atomicity/comment_2_381192724912dd814a2f5ce7daba74b1._comment
@@ -0,0 +1,46 @@
+[[!comment format=mdwn
+ username="http://richard.maw.name/"
+ nickname="Richard Maw"
+ avatar="http://cdn.libravatar.org/avatar/4d312a0989b085a9dfa9ba61d60da130979b9c408659132fea11d5efe2c90fc5"
+ subject="Re: Bugs in open_tmpfile()"
+ date="2016-11-28T23:13:53Z"
+ content="""
+Sorry I didn't notice your reply sooner.
+
+>   Firstly, it isn't const-correct:
+>   
+>   
+>       test.c: In function ‘open_tmpfile’:
+>       test.c:12:31: warning: passing argument 1 of ‘__xpg_basename’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
+>            strcat(template, basename(target));
+>                                      ^~~~~~
+>       In file included from test.c:1:0:
+>       /usr/include/libgen.h:34:14: note: expected ‘char *’ but argument is of type ‘const char *’
+>        extern char *__xpg_basename (char *__path) __THROW;
+>                     ^~~~~~~~~~~~~~
+
+This is from the awkwardness of there being two different definitions of basename.
+I don't get this error in the full version of the code (<http://yakking.branchable.com/posts/moving-files-8-atomicity/my-mv.c>)
+since I need to pull in the definitions with:
+
+    #include <libgen.h>          /* dirname */
+    #undef basename
+    #include <string.h>          /* basename */
+
+If you are aware of a better way to pull in the definitions please enlighten me, I'm working from the manpages as to how to pull the definitions in.
+
+>   Second, `strcpy(template, dirname(template));` has undefined behaviour since `dirname(template)` will probably point to template and `strcpy()` does not work with overlapping buffers. I think you have to use `memmove()` or two separate buffers.
+
+Thanks, I apparently missed the warning about overlaps when I checked the documentation.
+Given I only actually want to do any copy when `dirname(template)` doesn't point to `template` I've gone with:
+
+    char *dir = NULL;
+    strcpy(template, target);
+    dir = dirname(template);
+    if (dir != template)
+        strcpy(template, dir);
+
+>   Finally, it leaks the template string if `mkstemp()` fails, though that's not very likely.
+
+Thanks. I have no excuse for how I missed that, I was probably writing late into the night and then didn't go back and check.
+"""]]

Fix bugs in example thanks to bwh.
diff --git a/posts/moving-files-8-atomicity.mdwn b/posts/moving-files-8-atomicity.mdwn
index 64ef5de..462d6e5 100644
--- a/posts/moving-files-8-atomicity.mdwn
+++ b/posts/moving-files-8-atomicity.mdwn
@@ -24,9 +24,12 @@ and using `rename_file` after the contents have been changed.
 [[!format C """
 int open_tmpfile(const char *target, char **tmpfn_out) {
     char *template = malloc(strlen(target) + sizeof("./.tmpXXXXXX"));
+    char *dir = NULL;
     int ret;
     strcpy(template, target);
-    strcpy(template, dirname(template));
+    dir = dirname(template);
+    if (dir != template)
+        strcpy(template, dir);
     strcat(template, "/");
     strcat(template, ".tmp");
     strcat(template, basename(target));
@@ -34,6 +37,8 @@ int open_tmpfile(const char *target, char **tmpfn_out) {
     ret = mkstemp(template);
     if (ret >= 0)
         *tmpfn_out = template;
+    else
+        free(template);
     return ret;
 }
 
diff --git a/posts/moving-files-8-atomicity/my-mv.c b/posts/moving-files-8-atomicity/my-mv.c
index c6c7829..ebbfc46 100644
--- a/posts/moving-files-8-atomicity/my-mv.c
+++ b/posts/moving-files-8-atomicity/my-mv.c
@@ -737,9 +737,12 @@ cleanup:
 
 int open_tmpfile(const char *target, char **tmpfn_out) {
     char *template = malloc(strlen(target) + sizeof("./.tmpXXXXXX"));
+    char *dir = NULL;
     int ret;
     strcpy(template, target);
-    strcpy(template, dirname(template));
+    dir = dirname(template);
+    if (dir != template)
+        strcpy(template, dir);
     strcat(template, "/");
     strcat(template, ".tmp");
     strcat(template, basename(target));
@@ -747,6 +750,8 @@ int open_tmpfile(const char *target, char **tmpfn_out) {
     ret = mkstemp(template);
     if (ret >= 0)
         *tmpfn_out = template;
+    else
+        free(template);
     return ret;
 }
 
@@ -863,6 +868,7 @@ xdev:
     ret = unlink(source);
     if (ret < 0)
         perror("unlink");
+    return ret;
 }
 
 void strip_trailing_slashes(char *s) {

publishing
diff --git a/posts/software-undesign.mdwn b/posts/software-undesign.mdwn
new file mode 100644
index 0000000..5546c45
--- /dev/null
+++ b/posts/software-undesign.mdwn
@@ -0,0 +1,30 @@
+[[!meta date="Wed, 23 Nov 2016 12:00:06 +0000"]]
+[[!meta title="Software un-design"]]
+[[!meta author="Daniel Silverstone"]]
+
+We're surrounded by people telling us that software design comes from
+requirements, and that code comes from design.  However in the real world,
+particularly in hobby coding, it's very common for software to grow
+"organically" from the particular itches one or two people choose to scratch.
+
+The result of this is that software, particularly at the surface, is almost
+entirely there to satisfy its contributors.  Fortunately for the majority of us
+that means that early users of the software get to shape it which might mean
+that it meets our needs too.  The better software is written by people who
+actually spend time working out what their users need and writing for that,
+rather than for themselves.
+
+This issue can go beneath the surface too.  It's very common to find that
+internal APIs in software projects exist only to satisfy the author and thus
+the code being written.  It's rare to find a more complete API than the author
+themselves needs, unless the author is designing an API for others to use (and
+frankly even then it can be patchy).  This is so common that even when you're
+aware that it happens you can fall into the trap yourself.
+
+For this week, I'd like you to go back to some of your software which you have
+released (or if you haven't released any, some software you might one day want
+to release) and look at its functionality.  Consider if there might be gaps
+which users would like to be filled, and look at making the codebase coherent
+inside and out.  Then file some bugs, make some kanban cards, or whatever you
+need to let yourself know what you need to do.  If you're feeling energetic
+then make the changes too.

Added a comment: Bugs in open_tmpfile()
diff --git a/posts/moving-files-8-atomicity/comment_1_ec5c063c4b8b5ef386e7c26e05149e39._comment b/posts/moving-files-8-atomicity/comment_1_ec5c063c4b8b5ef386e7c26e05149e39._comment
new file mode 100644
index 0000000..dd2ceeb
--- /dev/null
+++ b/posts/moving-files-8-atomicity/comment_1_ec5c063c4b8b5ef386e7c26e05149e39._comment
@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="http://womble2.livejournal.com/"
+ subject="Bugs in open_tmpfile()"
+ date="2016-11-17T02:45:18Z"
+ content="""
+Firstly, it isn't const-correct:
+
+    test.c: In function ‘open_tmpfile’:
+    test.c:12:31: warning: passing argument 1 of ‘__xpg_basename’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
+         strcat(template, basename(target));
+                                   ^~~~~~
+    In file included from test.c:1:0:
+    /usr/include/libgen.h:34:14: note: expected ‘char *’ but argument is of type ‘const char *’
+     extern char *__xpg_basename (char *__path) __THROW;
+                  ^~~~~~~~~~~~~~
+
+Second, `strcpy(template, dirname(template));` has undefined behaviour since `dirname(template)` will probably point to `template` and `strcpy()` does not work with overlapping buffers.  I think you have to use `memmove()` or two separate buffers.
+
+Finally, it leaks the template string if `mkstemp()` fails, though that's not very likely.
+
+"""]]

publishing
diff --git a/posts/moving-files-8-atomicity.mdwn b/posts/moving-files-8-atomicity.mdwn
new file mode 100644
index 0000000..64ef5de
--- /dev/null
+++ b/posts/moving-files-8-atomicity.mdwn
@@ -0,0 +1,151 @@
+[[!meta date="Wed, 16 Nov 2016 12:00:07 +0000"]]
+[[!meta title="How difficult is it to move a file atomically?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux atomic]]
+
+# So now we've got an equivalent to a slow [rename(2)][], right?
+
+Not quite, [rename(2)][] is atomic.
+It disappears from the old location and reappears whole at the new one at the same time.
+
+Our move leaves it partially in the new location while it's copying
+which can confuse another program that might be looking at it
+while it is being copied.
+
+## Atomic creation with a temporary file
+
+This approach was described in a previous article.
+
+The bulk of the change is to include the clobber options,
+the rest is to open a temporary file in the target's directory
+(as seen in `open_tmpfile`)
+and using `rename_file` after the contents have been changed.
+
+[[!format C """
+int open_tmpfile(const char *target, char **tmpfn_out) {
+    char *template = malloc(strlen(target) + sizeof("./.tmpXXXXXX"));
+    int ret;
+    strcpy(template, target);
+    strcpy(template, dirname(template));
+    strcat(template, "/");
+    strcat(template, ".tmp");
+    strcat(template, basename(target));
+    strcat(template, "XXXXXX");
+    ret = mkstemp(template);
+    if (ret >= 0)
+        *tmpfn_out = template;
+    return ret;
+}
+
+int copy_file(char *source, char *target, struct stat *source_stat,
+              enum clobber clobber, enum setgid setgid, int required_flags) {
+    int srcfd = -1;
+    int tgtfd = -1;
+    int ret = -1;
+    char *tmppath = NULL;
+
+    ret = open(source, O_RDONLY);
+    if (ret == -1) {
+        perror("Open source file");
+        goto cleanup;
+    }
+    srcfd = ret;
+
+    ret = set_selinux_create_context(target, source_stat->st_mode);
+    if (ret != 0) {
+        perror("Set selinux create context");
+        goto cleanup;
+    }
+
+    ret = open_tmpfile(target, &tmppath);
+    if (ret == -1) {
+        perror("Open temporary target file");
+        goto cleanup;
+    }
+    tgtfd = ret;
+
+    ret = copy_contents(srcfd, tgtfd);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = fchmod(tgtfd, source_stat->st_mode);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = fix_owner(target, source_stat, setgid, tgtfd);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = copy_flags(srcfd, tgtfd, required_flags);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = copy_xattrs(srcfd, tgtfd);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = copy_posix_acls(srcfd, tgtfd);
+    if (ret < 0)
+        goto cleanup;
+
+    {
+        struct timespec times[] = { source_stat->st_atim, source_stat->st_mtim, };
+        ret = futimens(tgtfd, times);
+        if (ret < 0)
+            goto cleanup;
+    }
+
+    ret = rename_file(tmppath, target, clobber);
+cleanup:
+    close(srcfd);
+    close(tgtfd);
+    if (tmppath && ret != 0)
+        (void)unlink(tmppath);
+    free(tmppath);
+    return ret;
+}
+"""]]
+
+## Atomic creation with an anonymous temporary file
+
+Astute readers may have noticed that if your program crashes before renaming,
+that the temporary file may be left behind.
+
+A potential solution to this is to use the `O_TMPFILE` option of [open(2)][]
+and then [linkat(2)][] to link the temporary file into place.
+
+This is handy, because the file is cleared up automatically
+when the file descriptor is closed.
+
+However an issue with this approach
+is that [linkat(2)][] always returns `EEXIST` if the target exists,
+so if you wanted to clobber a file you would need to fall back
+to the temporary file approach as above,
+by using [linkat(2)][] to link the file in at a temporary path
+and then use [renameat2(2)][] to put it in place,
+which loses the primary benefit of the [linkat(2)][] in the first place.
+
+This may be worthwhile as it reduces the period
+where the file may be left behind,
+but such an approach is left as an exercise to the reader.
+
+As usual, the code may be downloaded from [[my-mv.c|my-mv.c]]
+and the accompanying [[Makefile|Makefile]] may be downloaded.
+
+# So we've got something we can use to move any file now?
+
+If you define "file" as a regular file, yes.
+
+If you say "everything is a file",
+then you open it up to character devices, block devices, unix sockets,
+symbolic links, directories and btrfs subvolumes.
+
+However every enterprise needs a given definition of done.
+
+For now I think copying a regular file is enough,
+so detail in other types of file will have to be left for a future series.
+
+[rename(2)]: http://man7.org/linux/man-pages/man2/rename.2.html
+[open(2)]: http://man7.org/linux/man-pages/man2/open.2.html
+[linkat(2)]: http://man7.org/linux/man-pages/man2/linkat.2.html
+[renameat2(2)]: http://man7.org/linux/man-pages/man2/renameat2.2.html
diff --git a/posts/moving-files-8-atomicity/Makefile b/posts/moving-files-8-atomicity/Makefile
new file mode 100644
index 0000000..92aab6d
--- /dev/null
+++ b/posts/moving-files-8-atomicity/Makefile
@@ -0,0 +1,15 @@
+
+define checkdef
+`if echo 'int main(){(void)$(1);}' | \
+    gcc -include stdio.h -xc - -o/dev/null 2>/dev/null; \
+ then \
+     echo 1; \
+ else \
+     echo 0; \
+ fi`
+endef
+
+my-mv: CFLAGS=-std=gnu99 -g -D_GNU_SOURCE -DHAVE_RENAMEAT2=$(call checkdef,renameat2) -DHAVE_COPY_FILE_RANGE=$(call checkdef,copy_file_range)
+my-mv: LDLIBS=-lselinux
+
+clobbering: CFLAGS=-D_GNU_SOURCE -DHAVE_RENAMEAT2=$(call checkdef,renameat2)
diff --git a/posts/moving-files-8-atomicity/my-mv.c b/posts/moving-files-8-atomicity/my-mv.c
new file mode 100644
index 0000000..c6c7829
--- /dev/null
+++ b/posts/moving-files-8-atomicity/my-mv.c
@@ -0,0 +1,999 @@
+
+#include <assert.h>
+#include <limits.h>          /* SSIZE_MAX */
+#include <stdbool.h>         /* bool */
+#include <stdio.h>           /* perror */
+#include <sys/types.h>       /* off_t, ssize_t, mode_t */
+#include <unistd.h>          /* read, write, lseek, SEEK_{SET,CUR,DATA,HOLE} */
+#include <fcntl.h>           /* open, splice */
+#include <errno.h>           /* E* */
+#include <sys/syscall.h> 
+#include <getopt.h>          /* getopt_long, struct option */
+#include <linux/btrfs.h>     /* BTRFS_IOC_CLONE */
+#include <sys/vfs.h>         /* ftatfs, struct statfs */
+#include <sys/stat.h>        /* struct stat, futimens */
+#include <sys/ioctl.h>       /* ioctl */
+#include <linux/magic.h>     /* BTRFS_SUPER_MAGIC */

(Diff truncated)
publishing
diff --git a/posts/other-peoples-timescales.mdwn b/posts/other-peoples-timescales.mdwn
new file mode 100644
index 0000000..e6dfa7d
--- /dev/null
+++ b/posts/other-peoples-timescales.mdwn
@@ -0,0 +1,28 @@
+[[!meta date="Wed, 09 Nov 2016 12:00:07 +0000"]]
+[[!meta title="Dancing to someone else's tune"]]
+[[!meta author="Daniel Silverstone"]]
+
+When you write software in isolation, for yourself and no others, you are not
+beholden to anyone's timescales but your own.  Sadly for those of us who enjoy
+writing software this is a very rare state of being for a project.  Usually
+there are other users, other developers, and other timescales to consider.
+
+In the world of free software, some projects make a thing out of aggregating a
+lot of software together into something larger and coherent.  We often refer to
+these things as 'Distributions' though there are other mechanisms by which
+software is aggregated.
+
+When you have some software which goes into a software distribution of some
+kind, you're suddenly beholden to someone else's timing.  For example if your
+software goes into a Debian release, you can be confident that the version of
+the program which goes into the release will be in active use for a number of
+years after the release; and you ought to provide support for that.  Sadly it
+is exactly this which encourages people to not have their software packaged
+into distributions; which results in more pain for users.
+
+If you love your users, I encourage you to pick at least one distribution and
+get your software into it.  And then set yourself up to be the best upstream
+you possibly can, and provide support for the duration of that distribution's
+release.  While it is hard, it is a surprisingly gratifying thing to do and
+will teach you lessons which no amount of Yakking articles could do.
+

creating tag page tags/tempfile
diff --git a/tags/tempfile.mdwn b/tags/tempfile.mdwn
new file mode 100644
index 0000000..6293161
--- /dev/null
+++ b/tags/tempfile.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged tempfile"]]
+
+[[!inline pages="tagged(tempfile)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/atomic-file-creation-tmpfile.mdwn b/posts/atomic-file-creation-tmpfile.mdwn
new file mode 100644
index 0000000..afc7e93
--- /dev/null
+++ b/posts/atomic-file-creation-tmpfile.mdwn
@@ -0,0 +1,173 @@
+[[!meta date="Wed, 02 Nov 2016 12:00:07 +0000"]]
+[[!meta title="Atomic file creation with temporary files"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux atomic tempfile]]
+
+We previously spoke about atomic file clobbering
+with [open(2)][] and [renameat2(2)][].
+
+This can be used to perform an atomic file creation too.
+
+At a really high level the idea is:
+
+1.  Create a temporary file.
+2.  Complete setting it up.
+3.  Rename the temporary file into place.
+
+In more detail:
+
+1.  Decide whether creating a new file or replacing an existing one.
+
+    This determines which flags [renameat2(2)][] gets later.
+    You can [stat(2)][] the destination path before proceeding
+    which allows you to detect whether the target was added or removed
+    while you were building the file.
+
+2.  Pick a temporary file path.
+
+    This needs to be on the same file system as the destination,
+    since we are going to use [renameat2(2)][] later.
+
+    Ideally this would be in a directory that is cleaned up automatically
+    so that if your program or computer crashes
+    you won't have the temporary file left around.
+
+    It's rare to be able to do this, since that pretty much just leaves
+    a creating files on a tmpfs when your `/tmp` is a mounted tmpfs,
+    or creating files on your root partition when `/tmp` is not a mounted tmpfs.
+
+    You could find the root directory of the mount point the destination is in
+    and put it in a temporary directory in there,
+    but this would still require system integration
+    to have it automatically cleaned up on mount.
+
+    With either `/tmp` or a custom tempdir on the mount point
+    you also have the problem that when you later use [renameat2(2)][]
+    it can fail from another process mounting a directory on top
+    of one of the directories in the destination path.
+
+    So even though it can leave a temporary file behind
+    creating your temporary file in the same directory as your target
+    is probably the best approach when atomically creating a file
+    with a temporary file.
+
+    [linkat(2)][] and `O_TMPFILE` provide a better solution for this,
+    but that's a topic for another article,
+    and only works for regular files.
+
+3.  Pick a random file path in your temporary directory.
+
+    This is what [mkstemp(3)][] does under the hood.
+
+    Since we intend to check whether the target file already exists
+    we could safely use [tmpnam(3)][] or [mktemp(3)][] to make it
+    (though not [tempnam(3)][] since it prefers the `TMPDIR`
+     environment variable for creating the temp file in
+     even if a directory is passed),
+    however in their zeal to stop people writing insecure programs
+    the POSIX standard has either removed or marked these functions obsolete,
+    so you probably want your own implementation.
+
+4.  Create your new file in a way that will fail if it already exists.
+
+    For regular files this requires passing `O_CREAT|O_EXCL` to the flags.
+
+    The system calls for creating
+    fifos, device nodes, symlinks, directories and unix sockets
+    already default to failing if the target already exists.
+
+    [open(2)][], [mkfifo(3)][], [mknod(2)][], [symlink(2)][] and [mkdir(2)][]
+    return an errno of `EEXIST`.
+
+    [bind(2)][] (for unix sockets) returns `EADDRINUSE`.
+
+    [open(2)][], [mkfifo(3)][], [mknod(2)][], [symlink(2)][] and [mkdir(2)][]
+    have variant syscalls with the "at" suffix
+    which take a file descriptor for the directory to create them in.
+
+    Using these or changing directory into the destination provides resilience
+    against the filesystem mounts changing mid-operation.
+
+    When using [bind(2)][] with a unix socket you probably want to [chdir(2)][]
+    anyway, since the maximum length path is shorter than `PATH_MAX`.
+
+    If creation fails, go back to step 4.
+
+5.  Complete initialisation of the file.
+
+    What this involves depends on the type of file and your application.
+
+    Regular files will want their contents written and metadata set.
+
+    Everything else probably just wants a few metadata tweaks.
+
+6.  Rename the file into place.
+
+    This is the [renameat2(2)][] trick mentioned in an earlier article.
+
+    1.  If you explicitly want to create the file pass `RENAME_NOREPLACE`.
+    2.  If you explicitly want to replace a file pass `RENAME_EXCHANGE`
+        and [unlink(2)][] the temporary file,
+    3.  If you're not sure [stat(2)][] the destination before creating
+        and pass `RENAME_NOREPLACE` if it didn't exist beforehand,
+        and `RENAME_EXCHANGE` as above if it did,
+        so if you get a failure you can warn, ask or abort.
+    4.  If you're really sure you don't care if it either creates a new file
+        or replaces an old one then don't pass any flags.
+
+    Few of the errors that may happen are recoverable from,
+    (though `EEXIST` when you don't care about clobbering can be handled
+     by removing the destination while linking fails)
+    but checking the return code can help you provide useful error messages.
+
+    *   `ENOENT` when using `RENAME_EXCHANGE` means the destination file
+        doesn't exist when it should.
+    *   `EEXIST` when using `RENAME_NOREPLACE` means the destination file
+        exists when it should not.
+    *   `EROFS` or `EXDEV` mean some process changed the file system mounts
+        while the operation was in-progress.
+        `EROFS` being making it read-only,
+        and `EXDEV` being some mount on top of your directory.
+
+    If you're particularly paranoid you can use [stat(2)][]
+    to check whether the destination file is still reachable
+    from the root directory
+    by checking whether the `st_dev` and `st_ino` match.
+
+And there you have it. Atomic file updates.
+
+# What would this be useful for
+
+This is useful if you have many processes that might make use of a file
+and you need to update it to a new version.
+
+This would be useful for shared caches (such as [/etc/ld.so.cache][ld.so(8)])
+which can't use a lock file to synchronise updates
+either for performance or legacy client reasons.
+
+It can also be used to atomically upgrade a service running on a unix socket,
+by starting the new version of the service on a temporary socket
+then renaming it into place, then closing and removing the old socket.
+
+With care this can be generalised up to a directory with no subdirectories
+by hard-linking the contents into a temporary directory,
+making whatever changes are necessary,
+then swapping directories and unlinking the temporary directory and contents.
+
+This doesn't work for subdirectories because you can't have a directory hardlink
+and if any of those subdirectories were in use
+you would get very different results if they were using that directory
+instead of your new version of that directory.
+
+[open(2)]: http://man7.org/linux/man-pages/man2/open.2.html
+[renameat2(2)]: http://man7.org/linux/man-pages/man2/renameat2.2.html
+[stat(2)]: http://man7.org/linux/man-pages/man2/stat.2.html
+[linkat(2)]: http://man7.org/linux/man-pages/man2/linkat.2.html
+[mkstemp(3)]: http://man7.org/linux/man-pages/man3/mkstemp.3.html
+[mkfifo(3)]: http://man7.org/linux/man-pages/man3/mkfifo.3.html
+[mknod(2)]: http://man7.org/linux/man-pages/man2/mknod.2.html
+[symlink(2)]: http://man7.org/linux/man-pages/man2/symlink.2.html
+[mkdir(2)]: http://man7.org/linux/man-pages/man2/mkdir.2.html
+[bind(2)]: http://man7.org/linux/man-pages/man2/bind.2.html
+[chdir(2)]: http://man7.org/linux/man-pages/man2/chdir.2.html
+[ld.so(8)]: http://man7.org/linux/man-pages/man8/ld.so.8.html

creating tag page tags/atomic
diff --git a/tags/atomic.mdwn b/tags/atomic.mdwn
new file mode 100644
index 0000000..0b0711b
--- /dev/null
+++ b/tags/atomic.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged atomic"]]
+
+[[!inline pages="tagged(atomic)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/create
diff --git a/tags/create.mdwn b/tags/create.mdwn
new file mode 100644
index 0000000..ddaec1c
--- /dev/null
+++ b/tags/create.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged create"]]
+
+[[!inline pages="tagged(create)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/clobbering.mdwn b/posts/clobbering.mdwn
new file mode 100644
index 0000000..5e1586c
--- /dev/null
+++ b/posts/clobbering.mdwn
@@ -0,0 +1,112 @@
+[[!meta date="Wed, 26 Oct 2016 11:00:12 +0000"]]
+[[!meta title="Atomically clobbering files"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename create C Linux atomic]]
+
+I mentioned atomicity at the end of my previous article
+and had intended to write about how to make all the previous operations atomic,
+but doing so requires understanding how to atomically clobber files
+before we can atomically move them.
+
+The naive approach would be to check whether the file existed
+before attempting the operation.
+
+However this runs the risk of the file being added or removed
+between the check and the operation,
+in what's known as a [TOCTOU][] attack.
+
+We previously mentioned how to atomically create a file without clobbering,
+but we may instead explicitly want to clobber it, or not care.
+
+For opening a file, this is just a matter of using different flags.
+
+[[!format C """
+enum clobber {
+    CLOBBER_PERMITTED     = 'p',
+    CLOBBER_REQUIRED      = 'R',
+    CLOBBER_FORBIDDEN     = 'N',
+    CLOBBER_TRY_REQUIRED  = 'r',
+    CLOBBER_TRY_FORBIDDEN = 'n',
+};
+
+int create_file(const char *path, mode_t mode, int flags,
+                enum clobber clobber) {
+    switch (clobber) {
+        case CLOBBER_PERMITTED:
+            flags |= O_CREAT;
+            break;
+        case CLOBBER_REQUIRED:
+        case CLOBBER_TRY_REQUIRED:
+            flags &= ~O_CREAT;
+            break;
+        case CLOBBER_FORBIDDEN:
+        case CLOBBER_TRY_FORBIDDEN:
+            flags |= O_CREAT|O_EXCL;
+            break;
+        default:
+            assert(0);
+    }
+    return open(path, flags, mode);
+}
+"""]]
+
+For renaming a file things get a bit more awkward.
+
+There are flags for changing how the [rename][rename(2)] behaves when the file exists,
+but there isn't one for requiring that it does so.
+
+Instead there's `RENAME_EXCHANGE`
+which will fail if the target does not exist
+and the source file will replace the target on success,
+but it has the side effect of leaving the target file behind where the source file was.
+
+This can be remedied by calling [unlink(2)][].
+
+[[!format C """
+int rename_file(const char *src, const char *tgt, enum clobber clobber) {
+    int ret = -1;
+    int renameflags = 0;
+
+    switch (clobber) {
+        case CLOBBER_REQUIRED:
+        case CLOBBER_TRY_REQUIRED:
+            renameflags = RENAME_EXCHANGE;
+            break;
+        case CLOBBER_FORBIDDEN:
+        case CLOBBER_TRY_FORBIDDEN:
+            renameflags = RENAME_NOREPLACE;
+            break;
+        default:
+            assert(0);
+    }
+
+    ret = renameat2(AT_FDCWD, src, AT_FDCWD, tgt, renameflags);
+    if (ret == 0) {
+        if (clobber == CLOBBER_REQUIRED || clobber == CLOBBER_TRY_REQUIRED) {
+            ret = unlink(src);
+        }
+        return ret;
+    }
+
+    if ((errno == ENOSYS || errno == EINVAL)
+        && (clobber != CLOBBER_REQUIRED
+            && clobber != CLOBBER_FORBIDDEN)) {
+        ret = rename(src, tgt);
+    }
+
+cleanup:
+    return ret;
+}
+"""]]
+
+
+A test program, 
+[[clobbering.c|clobbering.c]]
+and the accompanying [[Makefile|Makefile]] may be downloaded.
+
+This test program will rename if passed two file paths
+and copy standard input to a file if only passed one path.
+
+[rename(2)]: http://man7.org/linux/man-pages/man2/rename.2.html
+[unlink(2)]: http://man7.org/linux/man-pages/man2/unlink.2.html
+[TOCTOU]: https://en.wikipedia.org/wiki/Time_of_check_to_time_of_use
diff --git a/posts/clobbering/Makefile b/posts/clobbering/Makefile
new file mode 100644
index 0000000..19570f7
--- /dev/null
+++ b/posts/clobbering/Makefile
@@ -0,0 +1,12 @@
+
+define checkdef
+`if echo 'int main(){(void)$(1);}' | \
+    gcc -include stdio.h -xc - -o/dev/null 2>/dev/null; \
+ then \
+     echo 1; \
+ else \
+     echo 0; \
+ fi`
+endef
+
+clobbering: CFLAGS=-D_GNU_SOURCE -DHAVE_RENAMEAT2=$(call checkdef,renameat2)
diff --git a/posts/clobbering/clobbering.c b/posts/clobbering/clobbering.c
new file mode 100644
index 0000000..cb2b71a
--- /dev/null
+++ b/posts/clobbering/clobbering.c
@@ -0,0 +1,170 @@
+#include <assert.h>      /* assert */
+#include <errno.h>       /* errno, ENOSYS */
+#include <getopt.h>      /* struct option, getopt_long */
+#include <sys/types.h>   /* mode_t */
+#include <fcntl.h>       /* AT_*, O_*, open */
+#include <unistd.h>      /* close, unlink */
+#include <stdio.h>       /* rename* */
+#include <limits.h>      /* SSIZE_MAX */
+
+#if !HAVE_DECL_RENAMEAT2
+#include <sys/syscall.h> /* __NR_* */
+static inline int renameat2(int oldfd, const char *oldname, int newfd, const char *newname, unsigned flags) {
+    return syscall(__NR_renameat2, oldfd, oldname, newfd, newname, flags);
+}
+#endif
+
+#ifndef RENAME_NOREPLACE
+#define RENAME_NOREPLACE (1<<0)
+#endif
+#ifndef RENAME_EXCHANGE
+#define RENAME_EXCHANGE (1<<1)
+#endif
+
+enum clobber {
+    CLOBBER_PERMITTED     = 'p',
+    CLOBBER_REQUIRED      = 'R',
+    CLOBBER_FORBIDDEN     = 'N',
+    CLOBBER_TRY_REQUIRED  = 'r',
+    CLOBBER_TRY_FORBIDDEN = 'n',
+};
+
+int create_file(const char *path, mode_t mode, int flags,
+                enum clobber clobber) {
+    switch (clobber) {
+        case CLOBBER_PERMITTED:
+            flags |= O_CREAT;
+            break;
+        case CLOBBER_REQUIRED:
+        case CLOBBER_TRY_REQUIRED:
+            flags &= ~O_CREAT;
+            break;
+        case CLOBBER_FORBIDDEN:
+        case CLOBBER_TRY_FORBIDDEN:
+            flags |= O_CREAT|O_EXCL;
+            break;
+        default:
+            assert(0);
+    }
+    return open(path, flags, mode);
+}
+
+int rename_file(const char *src, const char *tgt, enum clobber clobber) {
+    int ret = -1;
+    int renameflags = 0;
+
+    switch (clobber) {
+        case CLOBBER_REQUIRED:
+        case CLOBBER_TRY_REQUIRED:

(Diff truncated)
creating tag page tags/selinux
diff --git a/tags/selinux.mdwn b/tags/selinux.mdwn
new file mode 100644
index 0000000..db23ebf
--- /dev/null
+++ b/tags/selinux.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged selinux"]]
+
+[[!inline pages="tagged(selinux)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/smack
diff --git a/tags/smack.mdwn b/tags/smack.mdwn
new file mode 100644
index 0000000..f9bd9c4
--- /dev/null
+++ b/tags/smack.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged smack"]]
+
+[[!inline pages="tagged(smack)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/moving-files-7-difficult-xattrs.mdwn b/posts/moving-files-7-difficult-xattrs.mdwn
new file mode 100644
index 0000000..b1a5ea5
--- /dev/null
+++ b/posts/moving-files-7-difficult-xattrs.mdwn
@@ -0,0 +1,176 @@
+[[!meta date="Wed, 19 Oct 2016 11:00:06 +0000"]]
+[[!meta title="How difficult is it to preserve extended attributes when moving a file?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux xattr security selinux smack]]
+
+# Dealing with semantically important [xattrs][attr(5)]
+
+We previously spoke about [extended attributes][attr(5)]
+like they were just another piece of metadata attached to files.
+
+However some have rather awkward interfaces as far as copying is concerned,
+some because they don't depend on the file's contents itself,
+and some because they are filesystem specific.
+
+## Selinux labels
+
+Selinux is a complicated mandatory access control mechanism.
+
+Rather than store the access control rules in the file,
+like POSIX ACLs do,
+the rules are stored elsewhere in the kernel
+and a reference to what kind of file it is,
+is stored in the file as an extended attribute
+called the "security label".
+
+The security label and the security context of the process accessing the file
+are looked up in the ACL rules in the kernel
+to determine whether the operation is permitted.
+
+The details of how to define Selinux rules is complicated
+and beyond the scope of this article.
+We only care how we should reapply the rules when moving the file.
+
+While we could copy the label from the old file into the new file,
+as we did for POSIX ACLs,
+Selinux contexts are defined by their file paths rather than the inodes,
+so after we move a file we should relabel it
+to what the file should have in the new location.
+
+Using [selinux_restorecon(3)][] might be tempting,
+but it leaves open a race condition
+where the file would be created with the wrong context
+so it temporarily accessible with the wrong label.
+
+If the file context should be preserved from the original file,
+then you must read the context from the extended attribute,
+either directly with [fgetxattr(2)][] or [fgetfilecon(3)][],
+and then set the context before creating the new file
+with [setfscreatecon(3)][].
+
+If instead it should have the label that the path database says it should be,
+then the required context can be found by using [selabel_open(3)][]
+with `SELABEL_CTX_FILE` to get a reference to the file contexts database,
+then getting the label it should have at that path using [selabel_lookup(3)][],
+and setting the context for new files with [setfscreatecon(3)][].
+
+Existing files can have their labels changed with [selinux_restorecon(3)][].
+
+The [setfscreatecon(3)][] API is unfortunate as it involves global state.
+Recent enough versions of Linux have the `O_TMPFILE` flag for file creation,
+which doesn't create a directory entry for the file when it is created,
+so you can modify the file before it is visible to other processes,
+and can be bound into place with [linkat(2)][].
+
+[[!format C """
+int set_selinux_create_context(const char *tgt, mode_t srcmode) {
+    int ret = 0;
+    struct selabel_handle *hnd = NULL;
+    char *context = NULL;
+
+    hnd = selabel_open(SELABEL_CTX_FILE, NULL, 0);
+    if (hnd == NULL) {
+        if (errno != ENOENT) {
+            ret = 1;
+        }
+        goto cleanup;
+    }
+
+    ret = selabel_lookup(hnd, &context, tgt, srcmode);
+    if (ret != 0) {
+        goto cleanup;
+    }
+
+    ret = setfscreatecon(context);
+
+cleanup:
+    freecon(context);
+    if (hnd != NULL)
+        selabel_close(hnd);
+    return ret;
+}
+"""]]
+
+## [SMACK](https://www.kernel.org/doc/Documentation/security/Smack.txt)
+
+This is another security technology.
+
+Like Selinux it has labels.
+These are stored in extended attributes matching `security.SMACK64*`,
+so require root privileges to copy faithfully.
+
+## btrfs flags
+
+Only worth copying if both source and destination are on btrfs,
+but if you then move a file back to btrfs you might want to restore them.
+
+The only flag of real interest is `"btrfs.compression"`,
+which is safe to ignore if moving to a file system which doesn't support it.
+
+A "brain dead" implementation for this and SMACK is to check the prefix,
+and silently accept failure if setting the attribute fails.
+
+[[!format C """
+static int copy_xattrs(int srcfd, int tgtfd) {
+    ssize_t ret;
+    char *names = NULL;
+    void *value = NULL;
+    size_t names_size = 0, value_size = 0;
+
+    ret = xattr_list(srcfd, &names, &names_size);
+    if (ret < 0)
+        goto cleanup;
+
+    for (char *name = names; name < names + names_size;
+         name = strchrnul(name, '\0') + 1) {
+        /* Skip xattrs that need special handling */
+        if (!str_starts_with(name, "user.") &&
+            !str_starts_with(name, "security.SMACK64") &&
+            !str_starts_with(name, "btrfs.")) {
+            continue;
+        }
+
+        ret = xattr_get(srcfd, name, &value, &value_size);
+        if (ret < 0)
+            goto cleanup;
+
+        ret = TEMP_FAILURE_RETRY(fsetxattr(tgtfd, name, value, value_size, 0));
+        if (ret < 0) {
+            if (errno == EINVAL &&
+                (str_starts_with(name, "security.SMACK64") ||
+                 str_starts_with(name, "btrfs."))) {
+                continue;
+            }
+            goto cleanup;
+        }
+    }
+
+cleanup:
+    free(names);
+    free(value);
+    return ret;
+}
+"""]]
+
+As with previous articles, the full version of the
+[[my-mv.c|my-mv.c]] source file and the [[Makefile|Makefile]]
+may be downloaded.
+
+The [[Makefile|Makefile]] has changed since earlier
+since it now needs to link against libselinux.
+
+# So now we've got an equivalent to a slow [rename(2)][], right?
+
+Not quite, [rename(2)][] is atomic.
+It disappears from the old location and reappears whole at the new one at the same time.
+
+
+[attr(5)]: http://man7.org/linux/man-pages/man5/attr.5.html
+[selinux_restorecon(3)]: http://man7.org/linux/man-pages/man3/selinux_restorecon.3.html
+[fgetxattr(2)]: http://man7.org/linux/man-pages/man2/fgetxattr.2.html
+[fgetfilecon(3)]: http://man7.org/linux/man-pages/man3/fgetfilecon.3.html
+[selabel_open(3)]: http://man7.org/linux/man-pages/man3/selabel_open.3.html
+[selabel_lookup(3)]: http://man7.org/linux/man-pages/man3/selabel_lookup.3.html
+[setfscreatecon(3)]: http://man7.org/linux/man-pages/man3/setfscreatecon.3.html
+[linkat(2)]: http://man7.org/linux/man-pages/man2/link.2.html
+[rename(2)]: http://man7.org/linux/man-pages/man2/rename.2.html
diff --git a/posts/moving-files-7-difficult-xattrs/Makefile b/posts/moving-files-7-difficult-xattrs/Makefile
new file mode 100644
index 0000000..877d0d1
--- /dev/null
+++ b/posts/moving-files-7-difficult-xattrs/Makefile
@@ -0,0 +1,15 @@
+
+define checkdef
+`if echo 'int main(){(void)$(1);}' | \
+    gcc -include stdio.h -xc - -o/dev/null 2>/dev/null; \
+ then \
+     echo 1; \
+ else \
+     echo 0; \
+ fi`
+endef
+
+my-mv: CFLAGS=-std=gnu99 -g -D_GNU_SOURCE -DHAVE_RENAMEAT2=$(call checkdef,renameat2) -DHAVE_COPY_FILE_RANGE=$(call checkdef,copy_file_range)

(Diff truncated)
publishing
diff --git a/posts/bug-handling.mdwn b/posts/bug-handling.mdwn
new file mode 100644
index 0000000..305bf93
--- /dev/null
+++ b/posts/bug-handling.mdwn
@@ -0,0 +1,68 @@
+[[!meta date="Wed, 12 Oct 2016 11:00:07 +0000"]]
+[[!meta title="Dealing with bugs against your project"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag bug]]
+
+It was late in the afternoon. The city was in that quiet period
+between the lunch time rush hour and the end of work day rush hour.
+The sun was getting low in the sky. My office blinds were making
+shafts of light that highlighted the dust particles hanging in the
+air. I was bored, but looking forward to an early evening date with a
+bottle of soda and a video stream to my couch.
+
+Suddenly, **blam**, a letter dropped into my INBOX. Oh, no! So much
+for my evening plans.
+
+I took my feet off my desk, and opened the missive. It was a report of
+the worst kind: someone's found a bug, and I would have to get off my
+gluteus maximus to squash it, and squash it good. I opened my drawer,
+and took out the key to the hardware cabinet, and strapped a .33 under
+my left armpit. (That's a .33 liter can of cola, not a gun. Using guns
+to deal with bugs would be ridiculous, what kind of a thug did you
+think I am?)
+
+-----
+
+So you have a free software project with users. Sooner or later
+someone will report a problem that they're experiencing, and this may
+turn out to be an actual bug in your program that you need to fix.
+Let's have a short look at how to do that well.
+
+Some recommendations, based on over three decades of writing software:
+
+* You should document how to report issues. Making unhappy users dig
+  for that information just makes them angry.
+
+* You should separate reports of issues separately from confirmed
+  bugs. A bug is something you need to fix, an issue is something you
+  need to investigate what causes it. An issue report is a discussion
+  to find out what is wrong, and it may or may not turn into a
+  confirmed bug that causes code changes.
+
+* Try to keep the list of open reports as short as possible, and also
+  the list of confirmed, but unfixed bugs. Long lists are
+  de-moralising, and also take effort to pick the next thing to pay
+  attention to. This is all waste.
+
+* It's good to have a public tracker of issues and bugs. It can be as
+  easy as a static web page you maintain manually, or it can be
+  something automated, such as the [Debian BTS][], or [Bugzilla][], or
+  any of the myriad other options.
+
+* Automated trackers tend to enforce some process. Some of them make
+  this rather heavy, others keep it quite lightweight. For small
+  projects, lightweight is a much better option. Only add a more
+  heavyweight process after it's already clearly needed.
+
+* You should probably make it easy to report issues. Some people
+  prefer to make it less easy in order to avoid too many issues being
+  reported for trivial things, but my preference is to keep the
+  threshold as low as possible.
+
+* Whatever you do, treat those who report issues with kindness.
+  [Be gracious][]. Be friendly. Be open to the possibility that you've
+  made a mistake.
+
+[Be gracious]: http://yakking.branchable.com/posts/be-gracious/
+[Debian BTS]: https://www.debian.org/Bugs/
+[Bugzilla]: https://bugzilla.mozilla.org/

creating tag page tags/xattr
diff --git a/tags/xattr.mdwn b/tags/xattr.mdwn
new file mode 100644
index 0000000..c4aca68
--- /dev/null
+++ b/tags/xattr.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged xattr"]]
+
+[[!inline pages="tagged(xattr)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/acl
diff --git a/tags/acl.mdwn b/tags/acl.mdwn
new file mode 100644
index 0000000..2efc245
--- /dev/null
+++ b/tags/acl.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged acl"]]
+
+[[!inline pages="tagged(acl)" actions="no" archive="yes"
+feedshow=10]]

creating tag page tags/security
diff --git a/tags/security.mdwn b/tags/security.mdwn
new file mode 100644
index 0000000..d42e5e2
--- /dev/null
+++ b/tags/security.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged security"]]
+
+[[!inline pages="tagged(security)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/moving-files-6-extended-attributes.mdwn b/posts/moving-files-6-extended-attributes.mdwn
new file mode 100644
index 0000000..c5a1f6b
--- /dev/null
+++ b/posts/moving-files-6-extended-attributes.mdwn
@@ -0,0 +1,206 @@
+[[!meta date="Wed, 05 Oct 2016 11:00:09 +0000"]]
+[[!meta title="How difficult is it to preserve extended attributes when moving a file?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux xattr security acl]]
+
+# Copying [extended attributes][attr(5)]
+
+[Extended attributes][attr(5)] allow you to provide extra metadata for a file.
+
+It's effectively a key-value store using a string for the key,
+but values can be arbitrary blobs.
+
+[flistxattr(2)][] to know which xattrs exist.
+[fgetxattr(2)][] to read any xattrs.
+[fsetxattr(2)][] to write an xattr to a file.
+
+Regular users can set [xattrs][attr(5)] beginning `user.`,
+and as far as Linux is concerned that's arbitrary data
+that it doesn't need to care what they are.
+
+However there are attributes outside the `user.` namespace
+which have special meaning to Linux,
+so we shouldn't try to copy everything as it is.
+
+[[!format C """
+static int realloc_double(void **buf, size_t *size) {
+    size_t new_size = *size * 2;
+    void *new_buf = realloc(*buf, new_size);
+    if (new_buf == NULL && new_size != 0)
+        return -1;
+    *buf = new_buf;
+    *size = new_size;
+    return 0;
+}
+
+static int xattr_list(int fd, char **names, size_t *size) {
+    ssize_t ret;
+
+    if (*names == NULL && *size == 0) {
+        ret = TEMP_FAILURE_RETRY(flistxattr(fd, NULL, 0));
+        if (ret < 0)
+            goto error;
+        *size = ret;
+
+        *names = malloc(*size);
+        if (*names == NULL) {
+            ret = -1;
+            goto error;
+        }
+    }
+
+    for (;;) {
+        ret = TEMP_FAILURE_RETRY(flistxattr(fd, *names, *size));
+        if (ret >= 0) {
+            *size = ret;
+            break;
+        }
+
+        if (errno != ERANGE)
+            goto error;
+
+        /* New xattr added since first flistxattr */
+        ret = realloc_double((void**)names, size);
+        if (ret < 0)
+            goto error;
+    }
+
+    ret = 0;
+error:
+    return ret;
+}
+
+static int xattr_get(int fd, const char *name, void **value, size_t *size) {
+    ssize_t ret;
+
+    if (*value == NULL && *size == 0) {
+        ret = TEMP_FAILURE_RETRY(fgetxattr(fd, name, NULL, 0));
+        if (ret < 0)
+            goto error;
+        *size = ret;
+
+        *value = malloc(*size);
+        if (*value == NULL) {
+            ret = -1;
+            goto error;
+        }
+    }
+
+    for (;;) {
+        ret = TEMP_FAILURE_RETRY(flistxattr(fd, *value, *size));
+        if (ret >= 0) {
+            *size = ret;
+            break;
+        }
+
+        if (errno != ERANGE)
+            goto error;
+
+        /* xattr grew since first getxattr */
+        ret = realloc_double(value, size);
+        if (ret < 0)
+            goto error;
+    }
+
+    ret = 0;
+error:
+    return ret;
+}
+
+static int str_starts_with(const char *s1, const char *s2) {
+    return strncmp(s1, s2, strlen(s2)) == 0;
+}
+
+static int copy_xattrs(int srcfd, int tgtfd) {
+    ssize_t ret;
+    char *names = NULL;
+    void *value = NULL;
+    size_t names_size = 0, value_size = 0;
+
+    ret = xattr_list(srcfd, &names, &names_size);
+    if (ret < 0)
+        goto cleanup;
+
+    for (char *name = names; name < names + names_size;
+         name = strchrnul(name, '\0') + 1) {
+        /* Skip xattrs that need special handling */
+        if (!str_starts_with(name, "user.")) {
+            continue;
+        }
+
+        ret = xattr_get(srcfd, name, &value, &value_size);
+        if (ret < 0)
+            goto cleanup;
+
+        ret = TEMP_FAILURE_RETRY(fsetxattr(tgtfd, name, value, value_size, 0));
+        if (ret < 0)
+            goto cleanup;
+    }
+
+cleanup:
+    free(names);
+    free(value);
+    return ret;
+}
+"""]]
+
+## POSIX ACLs
+
+Feature from a POSIX design specification that wasn't widely adopted,
+but Linux supports the draft specification.
+
+Not used much outside of NFS or SAMBA.
+You would be forgiven for not knowing they exist.
+
+How they work is beyond the scope of this article,
+but [man7.org][acl(5)] covers some details about what it does
+and [lwn.net][lwn-acl] covers some limitations.
+
+For this article we're not concerned with how to use POSIX ACLs,
+it's relevant to us because they work by storing data in a special attribute,
+so if we want to preserve this we need to copy `system.posix_acl_access`.
+
+Since this attribute doesn't start with `"user."`
+we need root privileges to copy it faithfully.
+
+[[!format C """
+int copy_posix_acls(int srcfd, int tgtfd) {
+    static const char name[] = "system.posix_acl_access";
+    int ret = 0;
+    void *value = NULL;
+    size_t size = 0;
+
+    ret = xattr_get(srcfd, name, &value, &size);
+    if (ret < 0) {
+        if (errno == ENODATA)
+            ret = 0;
+        goto cleanup;
+    }
+
+    ret = TEMP_FAILURE_RETRY(fsetxattr(tgtfd, name, value, size, 0));
+    if (ret < 0) {
+        goto cleanup;
+    }
+
+cleanup:
+    free(value);
+    return ret;
+}
+"""]]
+
+As with previous articles, the full version of the
+[[my-mv.c|my-mv.c]] source file and the [[Makefile|Makefile]]
+may be downloaded.
+

(Diff truncated)
creating tag page tags/flags
diff --git a/tags/flags.mdwn b/tags/flags.mdwn
new file mode 100644
index 0000000..5ec8119
--- /dev/null
+++ b/tags/flags.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged flags"]]
+
+[[!inline pages="tagged(flags)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/moving-files-5-flags.mdwn b/posts/moving-files-5-flags.mdwn
new file mode 100644
index 0000000..1b6cb54
--- /dev/null
+++ b/posts/moving-files-5-flags.mdwn
@@ -0,0 +1,268 @@
+[[!meta date="Wed, 28 Sep 2016 11:00:06 +0000"]]
+[[!meta title="How difficult is it to preserve flags when moving a file?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux flags]]
+
+[[Previously|moving-files-4-stat]] we spoke about the common,
+POSIX file metadata.
+
+This is not the only metadata that a program
+that handles copying files has to worry about on Linux.
+
+Files also have some additional flags for changing their behaviour,
+or possibly read-only flags for providing extra information.
+
+# Accessing flags.
+
+Flags were originally a feature of the `ext2` filesystem,
+which means they don't have a dedicated system call,
+since filesystem specific features are often implemented as [ioctls][ioctl(2)].
+
+It also explains why you might see it called `EXT2_IOC_GETFLAGS`
+or `EXT2_IOC_SETFLAGS`.
+
+When using ioctls, it's good to be paranoid,
+since the same ioctl number can be used for different devices,
+and you wouldn't want to accidentally do something unintended.
+
+It's possible to check whether it's an appropriate file
+by using stat and checking the file mode.
+
+We previously used this pattern for the file clone ioctl on btrfs,
+but included a check that it was a btrfs filesystem.
+
+Since file flags are applicable to multiple filesystems
+checking the filesystem type should not be necessary.
+
+[[!format C """
+#include <sys/stat.h>
+#include <errno.h>
+#include <linux/fs.h>
+#include <sys/ioctl.h>
+
+int get_flags(int fd, int *flags_out) {
+	struct stat st;
+	int ret = 0;
+	ret = fstat(fd, &st);
+	if (ret < 0)
+		return ret;
+	if (!S_ISREG(st.st_mode) && !S_ISDIR(st.st_mode)
+	    && !S_ISLNK(st.st_mode)) {
+		errno = ENOTTY;
+		return -1;
+	}
+	return ioctl(fd, FS_IOC_GETFLAGS, flags_out);
+}
+
+int set_flags(int fd, const int *flags) {
+	struct stat st;
+	int ret = 0;
+	ret = fstat(fd, &st);
+	if (ret < 0)
+		return ret;
+	if (!S_ISREG(st.st_mode) && !S_ISDIR(st.st_mode)
+	    && !S_ISLNK(st.st_mode)) {
+		errno = ENOTTY;
+		return -1;
+	}
+	return ioctl(fd, FS_IOC_SETFLAGS, flags);
+}
+"""]]
+
+As an aside,
+I find it odd that the set flags ioctl takes a `const int*` rather than an `int`
+since I know of no CPU that has shorter pointers than integers.
+
+# Copying flags
+
+Since filesystems have different capabilities,
+they unfortunately accept different sets of flags.
+
+[include/linux/fs.h][file-flags] has definitions for all the flags
+which are agreed on by every filesystem,
+though they may not support them.
+
+Since you can't trust flags on two filesystems to mean the same thing,
+if they are on different filesystems
+then you must attempt to only set flags they both agree on.
+
+[include/linux/fs.h][file-flags] defines `FS_FL_USER_MODIFIABLE` for this.
+
+Because filesystems may not implement every commonly defined flag
+and will refuse to set flags if any provided aren't recognised,
+you can either define logic for looking up the flags supported
+and setting those all at once,
+or try setting each flag in-turn
+so you can determine whether failing to copy that flag is a problem.
+
+Since the kernel doesn't expose which flags a filesystem supports at runtime
+the set of flags your program thinks a filesystem supports can get out of date,
+so setting the flags one at a time is the most flexible option.
+
+The code below uses [ffs(3)][] to iterate through the bits set in the integer
+since C doesn't have an operator to do it,
+but [ffs(3)][] may be a compiler builtin which uses special instructions.
+
+[[!format C """
+int copy_flags(int srcfd, int tgtfd, int required_flags) {
+    int ret;
+    int srcflags;
+    int tgtflags;
+    int newflags;
+    struct statfs srcfs, tgtfs;
+
+    ret = get_flags(srcfd, &srcflags);
+    if (ret != 0) {
+        /* If we don't support flags we have none to update. */
+        if (errno == EINVAL || errno == ENOTTY)
+            return 0;
+        return ret;
+    }
+
+    ret = get_flags(tgtfd, &tgtflags);
+    if (ret != 0) {
+        if (required_flags == 0 && (errno == EINVAL || errno == ENOTTY))
+            return 0;
+        return ret;
+    }
+
+    ret = fstatfs(srcfd, &srcfs);
+    if (ret != 0)
+        return ret;
+
+    ret = fstatfs(tgtfd, &tgtfs);
+    if (ret != 0)
+        return ret;
+
+    /* If on different fs need to mask to commonly agreed flags */
+    if (srcfs.f_type != tgtfs.f_type) {
+        srcflags &= FS_FL_USER_MODIFIABLE;
+        tgtflags &= FS_FL_USER_MODIFIABLE;
+        if ((srcflags & required_flags) != required_flags) {
+            errno = EINVAL;
+            return -1;
+        }
+    }
+
+    /* Skip setting flags if they are the same */
+    if (srcflags == tgtflags)
+        return 0;
+
+    /* Clear any flags that are set which we want to remove */
+    newflags = tgtflags & srcflags;
+    ret = set_flags(tgtfd, &newflags);
+    if (ret != 0) {
+        /* Can't set flags on the target, but we didn't require any. */
+        if (required_flags == 0 && errno == EINVAL)
+            return 0;
+        return ret;
+    }
+    tgtflags = newflags;
+
+    /* Use srcflags for flags we want to set,
+       which are everything not already set. */
+    srcflags &= ~tgtflags;
+    while (srcflags) {
+        int flag = 1 << (ffs(srcflags) - 1);
+
+        newflags = tgtflags | flag;
+        ret = set_flags(tgtfd, &newflags);
+        /* Fail if this flag is required and unsettable */
+        if (ret != 0 && (flag & required_flags))
+            return ret;
+        if (ret == 0)
+            tgtflags = newflags;
+
+        srcflags &= ~flag;
+    }
+
+    return 0;
+}
+"""]]
+
+# Driving it
+
+Since copying flags may or may not be a problem you need a way to decide,
+and, since that may depend on the context it's being called in,
+feedback may be more useful than a heuristic.
+
+A command-line application could ask for confirmation of
+whether it's acceptable to not set a flag,
+but this is awkward for programs used in batch scripts
+and you may have already noticed the above code uses `required_flags`
+so the user can declare which flags they consider essential.
+

(Diff truncated)
publishing
diff --git a/posts/be-liberal-and-strict.mdwn b/posts/be-liberal-and-strict.mdwn
new file mode 100644
index 0000000..75d6df3
--- /dev/null
+++ b/posts/be-liberal-and-strict.mdwn
@@ -0,0 +1,41 @@
+[[!meta date="Wed, 21 Sep 2016 11:00:08 +0000"]]
+[[!meta title="Be liberal in what you accept, and strict in what you produce"]]
+[[!meta author="Daniel Silverstone"]]
+
+There is a concept in computing which is known by various names, but perhaps
+the most common is the [Robustness Principle][].  It can be summarised as:
+
+> Be liberal in what you accept, and strict in what you produce.
+
+This is originally related to [TCP][] implementations but has since been
+applied to almost all higher level protocols over time.  The general concept is
+that when implementing a protocol you should be as careful as you can in what
+you generate, so that even the simplest of implementations can easily interpret
+your output; and that when interpreting what another implementation has sent to
+you, do your best to do something _useful_ with what you receive, even if it is
+not the most correct of messages.
+
+This is a good principle to work to, but like all good things, it can be taken
+to extremes; and in extremis it is a gun loaded and pointed at your head.  In
+general, being strict in what you produce can't really result in damage, but
+taking liberal acceptance to extremes is what causes confusing and unusual
+attacks being possible.  An excellent example of just how bad things can get
+when you're a little to liberal can be found at
+[Bouke van der Bijl's Blog](http://bouk.co/blog/hacking-developers/) related to
+attacking [Redis][], [Memcached][], and [Elasticsearch][] bound to localhost.
+
+A useful corollary to the robustness principle is [do not trust user input][].
+
+For your homework this week, go and have a look at something you have which
+processes input, generates output, and/or speaks any protocols.  (This ought to
+be almost any software you have written).  Look over that software's
+implementation of input data sanitisation or protocol implementations and
+decide if there's anything you could do to improve matters.  Then improve them
+and feel better about yourself for a little bit.
+
+[Robustness Principle]: https://en.wikipedia.org/wiki/Robustness_principle
+[TCP]: https://en.wikipedia.org/wiki/Transmission_Control_Protocol
+[Redis]: http://redis.io/
+[Memcached]: https://memcached.org/
+[Elasticsearch]: https://www.elastic.co/products/elasticsearch
+[do not trust user input]: https://msdn.microsoft.com/en-us/library/ee798441(v=cs.20).aspx

Add missing links to business model article
diff --git a/posts/business-models.mdwn b/posts/business-models.mdwn
index 5d4e041..a06e5c2 100644
--- a/posts/business-models.mdwn
+++ b/posts/business-models.mdwn
@@ -70,3 +70,7 @@ with short commentaries.
 As with everything else, making money out of free software is not
 automatic. If you want to run a business, software freedom is not a
 hindrance, and you'll still have to work hard to be successful.
+
+[Branchable]: http://www.branchable.com/
+[OpenSSL]: https://www.openssl.org/
+[git-annex]: http://git-annex.branchable.com/

creating tag page tags/business
diff --git a/tags/business.mdwn b/tags/business.mdwn
new file mode 100644
index 0000000..07c25a3
--- /dev/null
+++ b/tags/business.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged business"]]
+
+[[!inline pages="tagged(business)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/business-models.mdwn b/posts/business-models.mdwn
new file mode 100644
index 0000000..5d4e041
--- /dev/null
+++ b/posts/business-models.mdwn
@@ -0,0 +1,72 @@
+[[!meta date="Wed, 14 Sep 2016 11:00:07 +0000"]]
+[[!meta title="Software freedom business models"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag business]]
+
+Many free software developers would like to earn a living doing what
+they love. This is often thought to be tricky, since you can't just
+sell licences. Here's a list of business models for software freedom,
+with short commentaries.
+
+* **Support contracts** mean that users of the software, typically
+  business users, pay for support: for help solving any problems they
+  have with the software, such as fixing bugs, or helping with
+  deployment or problems during use. This is a very common, and
+  ideologically perfect business model.
+
+* **New feature development** is often combined with support
+  contracts. Some support issues are due to missing features, and some
+  users are happy to pay for their development. Also ideologically
+  pure.
+
+* **Consulting** is a generalisation of support and feature
+  development, and can also include giving advice on which software is
+  the best solution for the customer, and help integrating the
+  software with the customer's existing systems. Still ideologically
+  pure. 
+
+  Example for this and the models above: a myriad of companies
+  providing bespoke Linux kernel development.
+
+* **Training** is also a popular business model. New software is
+  frequently a bit scary to users, and it takes them a while to
+  properly up to speed. Training can make the transition easier, so
+  it's worth to some users. Ideologically pure.
+
+* **Hosting** is an option for some software, particularly services
+  accessed over the network, where the customer doesn't want to do the
+  hosting themselves. This might be hosting for a specific customer,
+  or for the general public. Ideologically pure as the driven snow.
+  Example: [Branchable][].
+
+* **Sponsorship** is a possible way to be paid, but is probably harder
+  to achive than an actual business. Sponsorship would probably come
+  from organisations who get a publicity boost from supporting the
+  project they sponsor. Example: [OpenSSL][] via the Linux Foundation
+  infrastructure project.
+
+* **Donations** and **crowd funding** are fashionable these days, but
+  somewhat difficult to be successful in. They probably mostly only
+  work for projects with a large number of users. For projects that
+  aren't ready to use yet, they'll probably only succeed if they
+  provide something that a lot of people want. Example: [git-annex][].
+
+* **Open core** is a name for the model where a project is free
+  software, but there's a proprietary version with extra features
+  (often called community vs enterprise versions). This is somewhat
+  iffy from a software freedom point of view: it is effectively a
+  proprietary software model with an limited open source version for
+  marketing purposes.
+
+* **Double licensing** is like open core, except there's no real
+  difference between the two versions. Paying customers just get the
+  software under a non-free license, which sometimes is preferable to
+  corporations than a free software license. Still iffy.
+
+* **Merchandise** can sometimes be a way of making a bit of extra on
+  the side, but rarely enough to live on. Examples are t-shirts,
+  stickers, and books. This is ideologically pure again.
+
+As with everything else, making money out of free software is not
+automatic. If you want to run a business, software freedom is not a
+hindrance, and you'll still have to work hard to be successful.

publishing
diff --git a/posts/be-gracious.mdwn b/posts/be-gracious.mdwn
new file mode 100644
index 0000000..c873bdd
--- /dev/null
+++ b/posts/be-gracious.mdwn
@@ -0,0 +1,32 @@
+[[!meta date="Wed, 07 Sep 2016 11:00:07 +0000"]]
+[[!meta title="Be gracious in how you accept"]]
+[[!meta author="Daniel Silverstone"]]
+
+We have, [[previously|upstreaming-eqiquette]] [[spoken|why-upstream]] about
+upstreaming changes and the etiquette associated with doing so.  Perhaps now
+is a good time to mention that the effort we recommend you to put forth when
+contributing to a project is also needed when you are the upstream.
+
+If you are fortunate enough to be upstream in a popular F/LOSS project then you
+may find yourself garnering contributions from the community you have fostered.
+This is a wonderful situation to be in, but it can also be a great burden on
+you.  We have mentioned that when you offer code to an upstream you are
+effectively offering a burden not a gift.  This means that as an upstream you
+need to recognise that the offer of a change is not something you must feel
+beholden to accept.  By the same token, if the person making the offer has
+managed to demonstrate their dedication to the community then you can usually
+trust that they'll help look after their contribution on an ongoing basis.
+
+Looking after a project as an upstream is often a thankless task and can become
+quite a grind to maintain a pleasant outlook in the face of the less
+experienced who may flock to you for assistance with your work.  Learning to be
+gracious in accepting any and all contributions which are not negative in
+nature can help you to foster a community of people who will then police
+themselves and assist you with the greater effort of looking after everything.
+
+Of course, you could just be a curmudgeonly upstream who acts as though the
+thought of anyone using their software is abhorrent.  That won't stop some
+intrepid users so be aware they might send you patches or suggestions anyway.
+Do try to not scare them too much.  If you do then they might fork your code
+and change it in unexpected ways; but with your name all over the codebase you
+might be inundated with questions anyway.

creating tag page tags/stat
diff --git a/tags/stat.mdwn b/tags/stat.mdwn
new file mode 100644
index 0000000..e061f11
--- /dev/null
+++ b/tags/stat.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged stat"]]
+
+[[!inline pages="tagged(stat)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/moving-files-4-stat.mdwn b/posts/moving-files-4-stat.mdwn
new file mode 100644
index 0000000..4817e99
--- /dev/null
+++ b/posts/moving-files-4-stat.mdwn
@@ -0,0 +1,369 @@
+[[!meta date="Wed, 31 Aug 2016 11:00:07 +0000"]]
+[[!meta title="How difficult is it to preserve metadata when moving a file?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux stat]]
+
+# So we've copied everything from the file now right?
+
+Well, we've copied all the data,
+and depending on your application that might be enough,
+but files also have metadata to worry about.
+
+We can see this by example,
+by checking the output of `ls -l`,
+before and after moving the file to a different filesystem.
+
+[[!format sh """
+$ touch /run/user/$(id -u)/testfile
+$ ls -l /run/user/$(id -u)/testfile
+-rw-rw-r-- 1 richardmaw richardmaw 0 Aug  8 19:40 /run/user/1000/testfile
+$ ./my-mv /run/user/$(id -u)/testfile testfile
+$ ls -l testfile
+-rw------- 1 richardmaw richardmaw 0 Aug  8 19:41 testfile
+"""]]
+
+You should be able to see that the `-rw-rw-r--` mode string,
+which represents readable for everyone and writable for the user and group,
+has become `-rw-------`, which represents read and write for the user only.
+
+This is because [ls(1)][] uses [stat(2)][],
+which is returning different data for the file.
+
+## Setting mode
+
+[stat(2)][] provided the mode of the file,
+in the `st_mode` field.
+
+[chmod(2)][] can be used set mode of the new file.
+
+The result of [stat(2)][] isn't exactly the same format as [chmod(2)][] takes,
+since in the [stat(2)][] field it includes bits saying what type of file it is,
+but [chmod(2)][] can't change what type a file is,
+so is only interested in the portion of the mode that is the permission bits.
+
+[[!format c """
+
+int copy_contents(int srcfd, int tgtfd) {
+    int ret = -1;
+    ret = btrfs_clone_contents(srcfd, tgtfd);
+    if (ret >= 0)
+        return ret;
+
+    if (ret < 0 && errno != EINVAL) {
+        /* Some error that wasn't from a btrfs clone,
+           so we can't fall back to something that would work */
+        perror("Copy file");
+        return -1;
+    }
+
+    ret = sparse_copy_contents(srcfd, tgtfd);
+    if (ret >= 0)
+        return ret;
+
+    if (ret < 0 && errno != EINVAL) {
+        /* Some error that wasn't from a sparse copy,
+           so we can't fall back to something that would work */
+        perror("Copy file");
+        return -1;
+    }
+
+    return naive_contents_copy(srcfd, tgtfd);
+}
+
+
+int copy_file(char *source, char *target, bool no_clobber) {
+    int srcfd = -1;
+    int tgtfd = -1;
+    int ret = -1;
+    struct stat source_stat;
+
+    ret = open(source, O_RDONLY);
+    if (ret == -1) {
+        perror("Open source file");
+        goto cleanup;
+    }
+    srcfd = ret;
+
+    ret = open(target, O_WRONLY|O_CREAT|(no_clobber ? O_EXCL : 0), 0600);
+    if (ret == -1) {
+        perror("Open target file");
+        goto cleanup;
+    }
+    tgtfd = ret;
+
+    ret = copy_contents(srcfd, tgtfd);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = fstat(srcfd, &source_stat);
+    if (ret < 0)
+        goto cleanup;
+
+    ret = fchmod(tgtfd, source_stat.st_mode);
+    if (ret < 0)
+        goto cleanup;
+cleanup:
+    close(srcfd);
+    close(tgtfd);
+    return ret;
+}
+
+"""]]
+
+## User and Group
+
+User and Group are numeric IDs
+that [ls(1)][] looks up in `/etc/passwd` and `/etc/group`
+to turn into a human readable name.
+
+The [chown(1)][] and [chgrp(1)][] take a name,
+but the [chown(2)][] system call does both using the numeric ID.
+
+The user and group can be found in the [stat(2)][] result
+in the `st_uid` and `st_gid` fields.
+
+### setgid bits
+
+If the `setgid` bit is set then newly created files
+have the group of the directory rather than the user that created them,
+but if files are moved in, then they have the group they had before.
+
+Depending on your application, it may make more sense to inherit the group
+or to preserve it from the original file.
+
+[[!format c """
+enum setgid {
+    SETGID_AUTO,
+    SETGID_NEVER,
+    SETGID_ALWAYS,
+};
+
+
+static int fix_owner(char *target, struct stat *source_stat, enum setgid setgid, int tgtfd) {
+    struct stat target_stat;
+    struct stat dirname_stat;
+    char *target_dirname;
+    int ret = 0;
+
+    if (setgid == SETGID_NEVER)
+        return fchown(tgtfd, source_stat->st_uid, source_stat->st_gid);
+
+    ret = fstat(tgtfd, &target_stat);
+    if (ret < 0) {
+        perror("Stat target file");
+        return ret;
+    }
+
+    target_dirname = dirname(target);
+    ret = stat(target_dirname, &dirname_stat);
+    if (ret < 0) {
+        perror("Stat target directory");
+        return ret;
+    }
+
+    if ((setgid == SETGID_ALWAYS
+         || (setgid == SETGID_AUTO && dirname_stat.st_gid & S_ISGID))
+        && target_stat.st_gid != dirname_stat.st_gid) {
+        ret = fchown(tgtfd, target_stat.st_uid, dirname_stat.st_gid);
+        if (ret < 0)
+            perror("Chown target");
+    }
+
+    return ret;
+}
+
+static int fix_rename_owner(char *target, struct stat *source_stat, enum setgid setgid) {
+    int tgtfd = -1;
+    int ret = -1;
+
+    ret = open(target, O_RDWR);
+    if (ret == -1) {
+        perror("Open target file");
+        goto cleanup;
+    }
+    tgtfd = ret;
+
+    ret = fix_owner(target, source_stat, setgid, tgtfd);
+cleanup:
+    close(tgtfd);
+    return ret;
+}
+
+int move_file(char *source, char *target, bool no_clobber, enum setgid setgid) {
+    int ret;
+    struct stat source_stat;

(Diff truncated)
publishing
diff --git a/posts/workflow-and-tools.mdwn b/posts/workflow-and-tools.mdwn
new file mode 100644
index 0000000..45d6300
--- /dev/null
+++ b/posts/workflow-and-tools.mdwn
@@ -0,0 +1,67 @@
+[[!meta date="Wed, 24 Aug 2016 11:00:07 +0000"]]
+[[!meta title="Workflow and tools"]]
+[[!meta author="Daniel Silverstone"]]
+
+This week will be a little bit of a reprise of some things I've spoken about
+before on the topic of [[IDEs|integrated-development-environments]] and some
+tangentially associated things.
+
+When we write software, we are using what some of the more flowery among us
+would like to call a [workflow][] but that most of us simply refer to as our
+[tools][].  We like to think that we use [best practice][] in our work too, and
+that the best practice actually is worth using.
+
+As I described before, there are some integrated development environments which
+aim to fulfil all the needs we have from our tools and in doing so, they often
+fix a particular workflow into place.  In addition, some tools are more
+powerful (or, rather, more flexible) than others and that can create friction
+when we try to combine techniques and tools to produce a workflow which suits
+us.
+
+As you may have guessed by now, I am writing about this topic because I was
+recently made exceedingly angry by incomplete integration between some very
+popular tools which I tried to make work together.  I was attempting to write
+some Java code and these days (apparently) the build system to use for Java
+apps is [Gradle][] and since I'm not a "native" Java programmer, I went with
+the recommendation.  I prepared my codebase, wrote my gradle file, built up a
+basic set of classes, got everything settled the way I wanted and then decided
+it was time to properly write some code.
+
+Now, in Java world, it's exceedingly rare to write code without the assistance
+of an IDE because, quite simply, Java is incredibly verbose in some places and
+IDEs make life a lot easier for dealing with the moment-to-moment boilerplate
+and refactoring pain.  One of the two major IDEs for Java is JetBrains'
+[IntelliJ IDEA][] and IDEA has integration for Gradle projects.  I fired up
+IDEA, pointed it at my `build.gradle` and it chugged away for a bit and then
+opened up a project workspace which looked perfect.  Sadly it turned out to be
+entirely worthless because while IDEA could read the Gradle file, examine my
+filesystem and decide what sources comprised my program, it was actually doing
+this in a way which didn't take into account that Gradle is in fact able to
+embed arbitrary Java and I happened to be using a feature of that which IDEA
+simply had no way of implementing.
+
+This resulted in a project which I could edit reasonably easily in IDEA, but
+which **had** to be built at the command line with gradle itself.  If I
+attempted to build and run the project inside IDEA, the code simply wouldn't
+work.  Since I had picked up this project on a whim, I'm sorry to say that I
+simply put it back down again, stepped away from the directory, and I doubt
+I'll be going back to it any time soon.  I **can** work around IDEA's inability
+to handle the feature I was using; but it'll be awkward and I am too annoyed at
+a pair of supposedly integrated tools entirely failing to do-the-right-thing.
+Sadly turing-complete build systems are intrinsically hard to reproduce without
+simply using the build system itself.
+
+The message I'm hoping you will take away from this little cautionary tale is
+that just because you're following what you think might be best practice and
+using tools which purport to function well together and support one another,
+you may find yourself hitting a brick wall within moments of wanting to
+actually produce some software.  I expect I'll get back to this project of mine
+at some point, and so perhaps also take away that it's okay to walk away from
+something which annoys you, and you can always come back to it later,
+especially if you remembered to commit it and push it to your Git server.
+
+[workflow]: https://en.wikipedia.org/wiki/Workflow
+[best practice]: https://en.wikipedia.org/wiki/Best_practice
+[tools]: https://en.wikipedia.org/wiki/Programming_tool
+[Gradle]: https://gradle.org/
+[IntelliJ IDEA]: https://www.jetbrains.com/idea/

publishing
diff --git a/posts/moving-files-3-faster.mdwn b/posts/moving-files-3-faster.mdwn
new file mode 100644
index 0000000..cc3b1f3
--- /dev/null
+++ b/posts/moving-files-3-faster.mdwn
@@ -0,0 +1,304 @@
+[[!meta date="Wed, 17 Aug 2016 11:00:06 +0000"]]
+[[!meta title="How difficult is it to move a file quickly?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux]]
+
+We previously made our file copy sparse-aware,
+so it only copies the data rather than the holes between the data,
+which as well as being more correct
+is also a lot faster for files which happen to be sparse.
+
+Files which are sparse also tend to be large,
+since they are usually some form of disk image,
+so while we are only copying the data there can still be a lot to copy
+so it would be convenient if we could to this more quickly.
+
+# [rename(2)][] was fast, can we make copying the data faster?
+
+That depends.
+
+[rename(2)][] and [link(2)][] get to be fast because they don't copy any data,
+they just change the directory metadata while keeping the data the same.
+
+Some filesystems have an extra level of indirection
+which lets multiple files share the same data though.
+
+## Cloning files with btrfs
+
+[btrfs][] and [ZFS][] are filesystems which support multiple files sharing data.
+
+Because of [ZFS][]' [interesting][fsf-zfs-on-linux] legal position
+I am more familiar with how [btrfs][] operates
+so I'm not going to talk about [ZFS][].
+
+If you want to copy a file between two btrfs file systems
+that happen to be stored on the same physical hard disks
+then you can "clone" the file's contents into the new file
+which is nearly as fast as a [rename(2)][].
+
+[[!format c """
+#include <linux/btrfs.h> /* BTRFS_IOC_CLONE */
+#include <sys/vfs.h>     /* ftatfs, struct statfs */
+#include <sys/stat.h>    /* struct stat */
+#include <sys/ioctl.h>   /* ioctl */
+#include <linux/magic.h> /* BTRFS_SUPER_MAGIC */
+
+int btrfs_clone_contents(int srcfd, int tgtfd) {
+	struct statfs stfs;
+	struct stat st;
+	int ret;
+
+	/* Behaviour is undefined unless called on a btrfs file,
+	   so ensure we're calling on the right file first. */
+	ret = fstatfs(tgtfd, &stfs);
+	if (ret < 0)
+		return ret;
+	if (stfs.f_type != BTRFS_SUPER_MAGIC) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	ret = fstat(tgtfd, &st);
+	if (ret < 0)
+		return ret;
+	if (!S_ISREG(st.st_mode)) {
+		errno = EINVAL;
+		return -1;
+	}
+
+	return ioctl(tgtfd, BTRFS_IOC_CLONE, srcfd);
+}
+
+int copy_file(const char *source, const char *target, bool no_clobber) {
+    int srcfd = -1;
+    int tgtfd = -1;
+    int ret = -1;
+    srcfd = open(source, O_RDONLY);
+    if (srcfd == -1) {
+        perror("Open source file");
+        return srcfd;
+    }
+    tgtfd = open(target, O_WRONLY|O_CREAT|(no_clobber ? O_EXCL : 0), 0600);
+    if (tgtfd == -1) {
+        perror("Open target file");
+        return tgtfd;
+    }
+
+    ret = btrfs_clone_contents(srcfd, tgtfd);
+    if (ret >= 0)
+        return ret;
+
+    if (ret < 0 && errno != EINVAL) {
+        /* Some error that wasn't from a btrfs clone,
+	   so we can't fall back to something that would work */
+        perror("Copy file");
+        return -1;
+    }
+
+    ret = sparse_copy_contents(srcfd, tgtfd);
+    if (ret >= 0)
+        return ret;
+
+    if (ret < 0 && errno != EINVAL) {
+        /* Some error that wasn't from a sparse copy,
+	   so we can't fall back to something that would work */
+        perror("Copy file");
+        return -1;
+    }
+
+    return naive_contents_copy(srcfd, tgtfd);
+}
+"""]]
+
+## What if I'm not using a filesystem that decouples files and their contents?
+
+You probably don't get the benefit of being able to share contents
+so you are unlikely to be able to copy a file without duplicating its data.
+
+It is however still possible to reduce the amount of effort involved.
+
+You may have noticed that the general pattern is read the contents into memory
+then write the contents from memory into the new file.
+
+This means that to copy a file you are copying the data twice,
+first out of the old file into your process' memory,
+then again into the new file.
+(The exact details are complicated.
+ If you've not opened the file with `O_DIRECT`
+ then the data may be cached so it's copied from kernel memory,
+ but if it's not then it's read from disk into kernel memory first.
+ When you write without `O_DIRECT` it queues up the data
+ to be written to disk at some point soon in the near future.)
+
+In some circumstances it may be possible to cut this down to one copy,
+straight from the source file into the target file.
+
+There are a handful of system calls for copying data
+from one file into another.
+
+Historically there has been [sendfile(2)][] and [splice(2)][]
+which copy data between two files
+without reading them into userspace first.
+
+[[!format c """
+#include <sys/sendfile.h> /* sendfile */
+ssize_t sendfile_copy_range(int srcfd, int tgtfd, size_t range) {
+    size_t to_copy = range;
+    while (to_copy) {
+        ssize_t ret = sendfile(tgtfd, srcfd, NULL, range);
+        if (ret < 0)
+            return ret;
+        to_copy -= ret;
+    }
+    return range;
+}
+
+ssize_t splice_copy_range(int srcfd, int tgtfd, size_t range) {
+    size_t to_copy = range;
+    while (to_copy) {
+        ssize_t ret = splice(srcfd, NULL, tgtfd, NULL, range, 0);
+        if (ret < 0)
+            return ret;
+        to_copy -= ret;
+    }
+    return range;
+}
+"""]]
+
+These were originally designed for speeding up web servers serving static files
+by copying data between files and pipes or sockets.
+
+[copy_file_range(2)][] was added to copy the contents of one file to another,
+after an a failed attempt to get it in under the name [reflink][],
+with the intention that it would be a filesystem independent way
+to share the contents of files like [btrfs][]'s clone ioctl.
+
+[[!format c """
+#if !HAVE_DECL_COPY_FILE_RANGE
+
+#ifndef __NR_copy_file_range
+#  if defined(__x86_64__)
+#    define __NR_copy_file_range 326
+#  elif defined(__i386__)
+#    define __NR_copy_file_range 377
+#  endif
+#endif
+
+static inline int copy_file_range(int fd_in, loff_t *off_in, int fd_out, loff_t *off_out, size_t len, unsigned int flags) {
+    return syscall(__NR_copy_file_range, fd_in, off_in, fd_out, off_out, len, flags);
+}
+#endif
+
+ssize_t cfr_copy_range(int srcfd, int tgtfd, size_t range) {
+    size_t to_copy = range;
+    while (to_copy) {

(Diff truncated)
publishing
diff --git a/posts/three-rs.mdwn b/posts/three-rs.mdwn
new file mode 100644
index 0000000..52ab10d
--- /dev/null
+++ b/posts/three-rs.mdwn
@@ -0,0 +1,39 @@
+[[!meta date="Wed, 10 Aug 2016 11:00:10 +0000"]]
+[[!meta title="Reduce, Reuse, Recycle"]]
+[[!meta author="Daniel Silverstone"]]
+
+There is a "popular" theme among environmentalists to encourage everyone to
+reduce waste, reuse items rather than throwing them away, and recycle items to
+new purpose wherever possible.  These three verbs can rather easily be applied
+to the world of software engineering, and the concepts have been the topic of
+a number of blog posts over time.
+
+---
+
+It's often a very hard thing for professional software engineers to say, but
+every line of code you produce is a cost not an asset.  The easiest to maintain
+code is something which was never written since it can never go wrong.  As
+such, simply reducing the *quantity* of code being written can be an effective
+way to improve the *quality* of software over all.
+
+As programmers we have the concept of code-reuse drummed into us almost from
+the get-go.  We're encouraged to use libraries which already exist, rather than
+reinventing the same thing again, and we're encouraged to abstract our code as
+much as is sensible to encourage its reuse within a project or even across
+projects.  By reusing code already written, we don't end up duplicating code
+which might have unseen subtleties, corner cases, or bugs.
+
+When a programmer writes a nice piece of code for solving a particular problem,
+it can sometimes be repurposed to solve another similar problem with very
+little effort.  This doesn't necessarily mean that the code in question would
+react well to being abstracted out into a library, but by recycling known-good
+code we can get a little closer to the ideal of having the least possible
+number of lines of code without losing the advantage of code already having
+been written.
+
+---
+
+The *programming is terrible* blog has a wonderful article on this, which I
+recommend that you all read and internalise.  It's called
+[Write code that is easy to delete, not extend](http://programmingisterrible.com/post/139222674273/write-code-that-is-easy-to-delete-not-easy-to)
+and while it isn't short, it's well worth it.

publishing
diff --git a/posts/moving-files-2-sparseness.mdwn b/posts/moving-files-2-sparseness.mdwn
new file mode 100644
index 0000000..edc5343
--- /dev/null
+++ b/posts/moving-files-2-sparseness.mdwn
@@ -0,0 +1,280 @@
+[[!meta date="Wed, 03 Aug 2016 11:00:07 +0000"]]
+[[!meta title="How difficult is it to move a sparse file?"]]
+[[!meta author="Richard Maw"]]
+[[!tag file rename move C Linux]]
+
+In our previous article,
+we established that if [rename(2)][] fails
+then we have to fall back to copying the file then removing the old one.
+
+We copied the data by just reading blocks from one file
+and writing them to the new one.
+
+For most files this would be sufficient,
+but for those that aren't this is a bad idea.
+
+# By reading and writing blocks the new file has exactly the same contents as the old one right?
+
+Not quite. Files can have "holes", where there is no data,
+but reading that range returns blocks of zeroes,
+so a naïve copy will produce a file that has no holes.
+
+This makes it take up more disk space,
+and other hole-aware software will treat it differently.
+
+An example would be a tool for writing disk images to disks,
+it could make things quicker by only writing the data in the disk images
+but if you created a copy without holes it would write a whole bunch of zeroes
+that it didn't need to
+and reduce the life of the hard-drive.
+
+To do this properly you need to either use [`FIEMAP`][],
+or [`SEEK_{DATA,HOLE}`][].
+
+While [`FIEMAP`][] is available in earlier kernels,
+since it exposes more information than is needed for copying a file
+and has historically been a source of disk corruption,
+we're going to proceed with [`SEEK_{DATA,HOLE}`][].
+
+# Copying sparsely with [lseek(2)][]
+
+The basis of the algorithm is to use [lseek(2)][] with `SEEK_HOLE`
+to find the end of a block of data and copy it,
+then use [lseek(2)][] with `SEEK_DATA` to find the end of the following hole.
+
+The difficulty, as always, is in the details.
+
+[[!format c """
+ssize_t copy_range(int srcfd, int tgtfd, size_t range);
+ssize_t naive_contents_copy(int srcfd, int tgtfd);
+
+ssize_t sparse_copy_contents(int srcfd, int tgtfd) {
+    size_t copied = 0;
+    off_t srcoffs = (off_t)-1;
+    off_t nextoffs = (off_t)-1;
+
+"""]]
+
+## Finding the start
+
+The first thing we need to do
+is to find out whether we started in a data block or a hole block.
+
+To do that we need to know where "here" is though,
+the way to do this is to call `lseek(fd, 0, SEEK_CUR)`,
+which logically means move the position forward 0 bytes,
+but has the side-effect of returning the current position.
+
+[[!format c """
+    srcoffs = TEMP_FAILURE_RETRY(lseek(srcfd, 0, SEEK_CUR));
+    if (srcoffs == (off_t)-1) {
+        perror("Find current position of file");
+        /* Can't seek file, could be file isn't seekable,
+           or that the current offset would overflow. */
+        return -1;
+    }
+"""]]
+
+## Starting with data or a hole?
+
+Now that we've got the current offset,
+we `lseek(fd, offset, SEEK_DATA)`.
+If the returned value is the same as the provided offset,
+then it was a data block,
+but if it moved then we were in a hole
+and it returned the start of the data.
+
+There's also the ever-present possibility that there is no more data,
+which sets [errno(3)][] to `ENXIO`.
+
+[[!format c """
+    nextoffs = TEMP_FAILURE_RETRY(lseek(srcfd, srcoffs, SEEK_DATA));
+    if (nextoffs == (off_t)-1) {
+        if (errno == ENXIO) {
+            /* NXIO means EOF, there is no data to copy,
+               but we may need to make a hole to the end of the file */
+            goto end_hole;
+        }
+        perror("Find data or hole at beginning of file");
+        /* Error seeking, must not support sparse seek */
+        return -1;
+    }
+
+    if (srcoffs != nextoffs)
+        /* Seeked to the end of a hole, can skip a data copy. */
+        goto hole;
+"""]]
+
+## Copying data and holes
+
+Depending on whether we started in data or in a hole,
+we either copy the contents of the data,
+or use [truncate(2)][] to extend the file without providing data.
+
+Because [truncate(2)][] does not advance the file offset,
+we have to use [lseek(2)][] to do it manually.
+
+As before, we can reach the end of the file when seeking,
+which breaks us out of the copy data then copy hole loop.
+
+[[!format c """
+    for (;;) {
+        ssize_t ret;
+        /* In data, so we must find the end of the data then copy it,
+           could pread/write. */
+        nextoffs = TEMP_FAILURE_RETRY(lseek(srcfd, srcoffs, SEEK_HOLE));
+        if (nextoffs == (off_t)-1) {
+            if (errno != ENXIO) {
+                perror("Find end of data");
+                return -1;
+            }
+
+            /* EOF after data, but we still need to copy */
+            goto end_data;
+        }
+
+        srcoffs = TEMP_FAILURE_RETRY(lseek(srcfd, srcoffs, SEEK_SET));
+        if (srcoffs == (off_t)-1) {
+            /* Rewinding failed, something is *very* strange. */
+            perror("Rewind back to data");
+            return -1;
+        }
+
+        ret = copy_range(srcfd, tgtfd, nextoffs - srcoffs);
+        if (ret < 0) {
+            return -1;
+        }
+        copied += ret;
+        srcoffs = nextoffs;
+
+        nextoffs = TEMP_FAILURE_RETRY(lseek(srcfd, srcoffs, SEEK_DATA));
+        if (nextoffs == (off_t)-1) {
+            if (errno == ENXIO) {
+                /* NXIO means EOF, there is no data to copy,
+                   but we may need to make a hole to the end of the file */
+                goto end_hole;
+            }
+            perror("Find end of hole");
+            /* Error seeking, must not support sparse seek */
+            return -1;
+        }
+hole:
+        /* Is a hole, extend the file to the offset */
+        ret = TEMP_FAILURE_RETRY(ftruncate(tgtfd, nextoffs));
+        if (ret < 0) {
+            perror("Truncate file to add hole");
+            return -1;
+        }
+
+        /* Move file offset for target to after the newly added hole */
+        nextoffs = TEMP_FAILURE_RETRY(lseek(tgtfd, nextoffs, SEEK_SET));
+        if (nextoffs == (off_t)-1) {
+            /* Something very strange happened,
+               either some race condition changed the file,
+               or the file is truncatable but not seekable
+               or some external memory corruption,
+               since EOVERFLOW can't happen with SEEK_SET */
+            perror("Move to after newly added hole");
+            return -1;
+        }
+
+        srcoffs = nextoffs;
+    }
+"""]]
+
+## Filling it to the end
+
+When finished with the copy-hole copy-data loop,
+we still have to fill the rest of the file,
+either by truncating it to fill in the final hole,
+or copying the rest of the data.
+
+[[!format c """
+end_hole:
+    nextoffs = TEMP_FAILURE_RETRY(lseek(srcfd, 0, SEEK_END));

(Diff truncated)
Fix typo (thanks, pabs)
diff --git a/posts/comparative-freeness.mdwn b/posts/comparative-freeness.mdwn
index e6c3903..476da52 100644
--- a/posts/comparative-freeness.mdwn
+++ b/posts/comparative-freeness.mdwn
@@ -44,7 +44,7 @@ customers are happy to pay you for support and feature development.
 A word of warning. Any discussions about these topics should be
 treated carefully. It's like using nitroglycerin: it's an important
 compound with several practical, industrial uses, but do not, repeat,
-DO NOT make juggling balls filled with it. If a dicussion about the
+DO NOT make juggling balls filled with it. If a discussion about the
 relative freedom of licence types becomes heated, step away. It's not
 worth participating anymore. The best case scenario is that several
 people get their feelings hurt and stop talking to each other,

creating tag page tags/freedom
diff --git a/tags/freedom.mdwn b/tags/freedom.mdwn
new file mode 100644
index 0000000..28b0b54
--- /dev/null
+++ b/tags/freedom.mdwn
@@ -0,0 +1,4 @@
+[[!meta title="pages tagged freedom"]]
+
+[[!inline pages="tagged(freedom)" actions="no" archive="yes"
+feedshow=10]]

publishing
diff --git a/posts/comparative-freeness.mdwn b/posts/comparative-freeness.mdwn
new file mode 100644
index 0000000..e6c3903
--- /dev/null
+++ b/posts/comparative-freeness.mdwn
@@ -0,0 +1,70 @@
+[[!meta date="Wed, 27 Jul 2016 11:00:07 +0000"]]
+[[!meta title="Which license is the most free?"]]
+[[!meta author="Lars Wirzenius"]]
+[[!tag licence freedom]]
+
+Free software licences can be roughly grouped into [permissive][] and
+[copyleft][] ones. Examples are the [BSD][] and [MIT][] licences for
+permissive and the [GNU GPL][] for copyleft. A common argument is
+about which one is more free.
+
+Permissive licences typically allow using a different licence for any
+derived works. In other words, if you release some software under a
+permissive licence, I can make changes to it and release the result
+under almost any other licence, including a proprietary, non-free one.
+
+A copyleft licence requires a derived work to be released under the
+same licence as the original. I can't make my own version and make it
+not be free (unless I get all copyright holders to agree to a licence
+change).
+
+This is the crux of the dispute of relative freeness. A permissive
+licence lets you do things that a copyleft one forbids, so clearly the
+permissive licence is more free. A copyleft licence means software
+using it won't ever become non-free against the wills of the copyright
+holders, so clearly a copyleft licence is more free than a permissive
+one.
+
+Both sides are both right and wrong, of course, which is why this
+argument will continue forever. Because there's no clear objective
+winner, the arguments easily get very, very hot.
+
+A related argument which type of license is better for doing business
+with. Many in favour of permissive licences claim that it's better for
+business, because you can do things in more traditional ways: sell
+non-free licences to customers, for example.
+
+However, making a living or running a business using a copyleft
+licence is certainly possible, and not even unusual. It may require
+coming up with a different business model. In fact, because almost all
+software development is really a service business anyway, it doesn't
+necessarily matter a whole lot what the licence is, as long as your
+customers are happy to pay you for support and feature development.
+
+A word of warning. Any discussions about these topics should be
+treated carefully. It's like using nitroglycerin: it's an important
+compound with several practical, industrial uses, but do not, repeat,
+DO NOT make juggling balls filled with it. If a dicussion about the
+relative freedom of licence types becomes heated, step away. It's not
+worth participating anymore. The best case scenario is that several
+people get their feelings hurt and stop talking to each other,
+possibly forever.
+
+If you want to make a choice between permissive and copyleft, you need
+to do it based on something else than their relative freeness. You
+might prefer permissive because it makes it easier to combine with
+other licences. You might choose copyleft because you don't want your
+software to ever become non-free. Or you might choose based on the
+most common licence type in the community you participate in. Or based
+on the length of the licence, or which one has a prettier SHA1 of the
+contents of the licence. And if anyone questions your choice, avoid
+getting into a fight.
+
+Disclaimer: This was written by someone who has chosen copyleft and
+has earned much of their adult salary writing copyleft software.
+
+[permissive]: https://en.wikipedia.org/wiki/Permissive_free_software_licence
+[copyleft]: https://en.wikipedia.org/wiki/Copyleft
+[GNU GPL]: https://www.gnu.org/licenses/gpl.html
+[MIT]: https://opensource.org/licenses/MIT
+[BSD]: https://opensource.org/licenses/BSD-2-Clause

publishing
diff --git a/posts/hacking-alone-hacking-together.mdwn b/posts/hacking-alone-hacking-together.mdwn
new file mode 100644
index 0000000..0b74a9d
--- /dev/null
+++ b/posts/hacking-alone-hacking-together.mdwn
@@ -0,0 +1,50 @@
+[[!meta date="Wed, 20 Jul 2016 11:00:08 +0000"]]
+[[!meta title="Hacking alone, hacking together"]]
+[[!meta author="Daniel Silverstone"]]
+
+Of late, I have been involved in a number of F/LOSS
+[hack days](https://en.wikipedia.org/wiki/Hackathon).  Most of them have been
+based around my own F/LOSS project ([Gitano](https://www.gitano.org.uk/)) and
+over the years several have been related to
+[NetSurf](http://www.netsurf-browser.org/).  One thing which characterises all
+these hack days is that they have been small (the largest was around six
+people).
+
+Another thing which characterises them is that they were done on a shoestring
+budget, with donated venues, donated tea and biscuits, and most attendees
+paying their own way there and back; and yet, without exception, they helped us
+get stuff done; often stuff which had been floundering for a while.
+
+Many hackers find it much easier to hack on their own.  From time to time I
+find that I do my best work when I can focus and not be distracted by anyone
+else; but most of the time I find that I do my best F/LOSS work when there's
+others on the project nearby to chat to about the problems we face.
+Often-times the hack day is perhaps better named a 'design day' where we get a
+lot of useful discussion and design done (e.g. the day when
+[Richard](/bio/richard) and I did the Gitano I18n design) and that's fine too.
+There's no hard and fast rule about what you get up to on a hack day.
+
+If you're involved in a smaller F/LOSS project then often there's no resources
+for big flashy conferences, or perhaps not enough clout to swing a dev-room
+somewhere like [FOSDEM](https://fosdem.org); but that doesn't mean there are
+not options available to you.  Naturally the smaller hackdays work best when
+the potential attendees are physically colocated, but that need not be a
+showstopper if it cannot be met.  Simply arranging a time when everyone
+involved in a project will agree to be on IRC, TeamSpeak, or any other
+communication tool which might be appropriate can result in an effective way to
+improve the state of a project.
+
+If you're involved in a somewhat larger project, then a resource such as the
+[Hackday manifesto](http://hackdaymanifesto.com/) may be of use to you; and if
+you happen to be part of a huge project then perhaps there'll be dedicated
+conferences for you to attend, such as [Debconf](http://www.debconf.org/).
+
+The moment there's more than you on a project, even if all you have for
+contributors are people who will try new versions and let you know how they
+work for them, you can make use of some level of hackday from time to time.
+
+Your homework is to go over your inventory of F/LOSS projects which you count
+yourself as at least somewhat involved with, look at where they might have had
+hackdays in the past, and where they might be planning them in the future, and
+then either get yourself along to a hackday, attend one on IRC, or if you're
+feeling super-enthusiastic, then propose to organise one yourself.