←	Nov 2024	→
S	M	T	W	T	F	S
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30

Detaching from parent environment
Instructing the daemon to terminate
Synchronous daemon start
Supporting fork-unsafe libraries
Shedding most of the code and using systemd notify

A daemon is a process that is always running in the background to provide some form of service. This is different from the nasal demons that may be invoked when you invoke undefined behaviour when writing a C program.

This form of daemon is named after Maxwell's demon, rather than having any diabolic undertones.

The tricky parts of starting a daemon are disassociating onesself from the process that launched it, since a great deal of context is inherited from the parent process; and providing notification when the service has started.

We're going to illustrate how this works with a python script that creates a daemon that writes the numbers 1 to 10 to a file, one every second before exiting.

Detaching from parent environment

#!/usr/bin/python
#test.py
import os, sys, time
filename = os.path.abspath(sys.argv[1])

# To disassociate ourself from our parent process we do a "double fork"
# the first fork being the fork that started our launcher script
daemonpid = os.fork()
if daemonpid == 0:
    # In the daemon process

    # Another thing we inherited from our parent process is our
    # "controlling tty", which can be accessed by opening /dev/tty. We want
    # to isolate ourself from our controlling tty so that a daemon we start
    # cannot access our terminal for input or output.
    # One way to do this is call setsid(), which gives us a new session.
    # Starting a new session also means that we no longer get signals
    # intended for all processes in the launcher session, or all processes
    # in our process group, as would be invoked by using kill() with a
    # negative process group ID.
    os.setsid()

    # Now we also inherited a lot of environment variables, so we're going
    # to clean them all out. Other daemons may need to preserve some
    # variables, but we don't need to.
    for var in os.environ:
        os.unsetenv(var)

    # There's also the umask(), which in inherited and affects what we set
    # the file permissions to by default.
    os.umask(0)

    # You also inherited a bunch of file descriptors. You should not have
    # any, so we must close them all. One rookie mistake is forgetting to
    # close the stdout, and having some debug output be produced by the
    # program. If this is left in, then the daemon will crash when the
    # user logs out, because its output file no longer exists.
    for pid in os.listdir('/proc/self/fd'):
        os.close(int(pid))

    # Because a lot of standard library functions assume that the standard
    # input, output and error are connected, we should replace them here.
    os.open('/dev/null', os.O_RDONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.open('/dev/null', os.O_WRONLY)

    # We should also change the current directory, as we will prevent it
    # from being unmounted otherwise.
    os.chdir('/')

    # Now we have disassociated ourselves from the majority of our
    # context, we can perform our operation
    # Note, we add the NOCTTY flag because we are accepting user input,
    # and if the user told us to open a tty, we would then have a
    # controlling tty again.
    fd = os.open(filename, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
    with os.fdopen(fd, 'w') as f:
        for i in xrange(1, 11):
            f.write('%s\n' % i)
            f.flush()
            time.sleep(1)
    sys.exit(0)

elif daemonpid > 0:
    # In the launcher process

    # Normally when a process dies, its parent process has to call some
    # form of wait() to get information about how it exited, so it can be
    # cleaned up.
    # It gets asynchronously notified with a SIGCHILD, which is annoying,
    # since we would have to have our launcher process live as long as our
    # daemon so we could clean it up.
    # However, if we exit here, then our daemon process is re-parented to
    # process 1, the "init" process, so it is owned by that, rather than
    # the process that started the "launcher" script.
    sys.exit(0)

Instructing the daemon to terminate

However, what if we want it to continue operating until we say it should stop? The traditional way to do this is to have the daemon write a "pid file" to say which process ID should be killed to stop the daemon.

#!/usr/bin/python
#test.py
import os, sys, time, itertools
filename = os.path.abspath(sys.argv[1])
pidfile = os.path.abspath(sys.argv[2])

daemonpid = os.fork()
if daemonpid == 0:
    os.setsid()
    for var in os.environ:
        os.unsetenv(var)
    os.umask(0)
    for pid in os.listdir('/proc/self/fd'):
        os.close(int(pid))
    os.open('/dev/null', os.O_RDONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.chdir('/')
    fd = os.open(filename, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)

    # As soon as we are ready, we write our daemon's pid to the pid file,
    # and not sooner, as we could leave around a pid file for a process
    # that failed to start, hence cannot be stopped, and some software may
    # opt to wait for the pid file to be written as a form of
    # notification.
    pidfd = os.open(pidfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
    with os.fdopen(pidfd, 'w') as pidfobj:
        pidfobj.write('%s\n' % os.getpid())

    with os.fdopen(fd, 'w') as f:
        for i in itertools.count():
            f.write('%s\n' % i)
            f.flush()
            time.sleep(1)
elif daemonpid > 0:
    sys.exit(0)

We can then, if we invoke the script as ./test.py test.out test.pid, run kill $(cat test.pid) to end the service.

Synchronous daemon start

However, this still claims to have successfully started the daemon before it has actually finished starting, and has no way of reporting whether starting actually failed. We alluded to possibly using the contents of the pid file as a signal for whether it started successfully, but that would require you to either poll the pid file or watch for writes to it with inotify, and also handle SIGCHILD if the daemon fails to start.

It's a lot easier to just open a pipe and use the end of file marker for whether the process is either ready or failed. This works because you get an end of file if the write-end is closed, which also happens if the process terminates.

#!/usr/bin/python
#test.py
import os, sys, time, itertools
filename = os.path.abspath(sys.argv[1])
pidfile = os.path.abspath(sys.argv[2])

# Here we create our pipe for status reporting, we want it in blocking mode
# so that our launcher process can sleep until it has status to report.
status_pipe_read, status_pipe_write = os.pipe()

daemonpid = os.fork()
if daemonpid == 0:

    # We don't _need_ to close it here as we close every unused file
    # later, but I've closed it here for illustration that we don't need
    # the reading end in the daemon process.
    os.close(status_pipe_read)

    os.setsid()
    for var in os.environ:
        os.unsetenv(var)
    os.umask(0)
    for pid in os.listdir('/proc/self/fd'):
        if int(pid) == status_pipe_write:
            continue
        os.close(int(pid))
    os.open('/dev/null', os.O_RDONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.chdir('/')
    fd = os.open(filename, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
    pidfd = os.open(pidfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
    with os.fdopen(pidfd, 'w') as pidfobj:
        pidfobj.write('%s\n' % os.getpid())
    with os.fdopen(fd, 'w') as f:

        # We must close the status pipe file descriptor now to signal eof
        os.close(status_pipe_write)

        for i in itertools.count():
            f.write('%s\n' % i)
            f.flush()
            time.sleep(1)
elif daemonpid > 0:

    # In the launcher process we *must* close the write end of the pipe, or
    # we would not get an end of file on the pipe when it is closed in the
    # daemon process.
    os.close(status_pipe_write)

    # Now we wait for status output
    status = os.read(status_pipe_read, 4096)
    assert status == ''

    # Now we must handle the exit status of the process
    deadpid, status = os.waitpid(daemonpid, os.WNOHANG)
    # If our daemon has prematurely exited, we use its exit code
    if os.WIFEXITED(status):
        ecode = os.WEXITSTATUS(status)
        if ecode != 0:
            sys.stderr.write('Daemon launch failed: %d\n' % ecode)
        sys.exit(ecode)
    else:
        sys.exit(0)

Supporting fork-unsafe libraries

However, there's some libraries that misbehave after a fork, perhaps because they use threads, and after fork, you only get a copy of the thread that called fork. To handle this appropriately you need to exec a helper binary after the fork, so you ought to split your daemon into launcher and payload binaries if you ever use such libraries.

#!/usr/bin/python
#launcher.py
import os, sys
filename = os.path.abspath(sys.argv[1])
pidfile = os.path.abspath(sys.argv[2])
payload = os.path.abspath('payload.py')
status_pipe_read, status_pipe_write = os.pipe()
daemonpid = os.fork()
if daemonpid == 0:
    os.setsid()
    for var in os.environ:
        os.unsetenv(var)
    os.umask(0)
    for pid in os.listdir('/proc/self/fd'):
        if int(pid) == status_pipe_write:
            continue
        os.close(int(pid))
    os.open('/dev/null', os.O_RDONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.open('/dev/null', os.O_WRONLY)
    os.chdir('/')

    # call out to our payload, with the protocol that the status fd is the
    # first argument, and the pid file path is the second
    os.execv(payload, [payload, str(status_pipe_write), pidfile, filename])

elif daemonpid > 0:
    os.close(status_pipe_write)
    status = os.read(status_pipe_read, 4096)
    assert status == ''
    deadpid, status = os.waitpid(daemonpid, os.WNOHANG)
    if os.WIFEXITED(status):
        ecode = os.WEXITSTATUS(status)
        if ecode != 0:
            sys.stderr.write('Daemon launch failed: %d\n' % ecode)
        sys.exit(ecode)
    else:
        sys.exit(0)

We also have a separate payload script, that should be located in the same directory we run the launcher from.

#!/usr/bin/python
#payload.py
import sys, os, time, itertools
status_pipe_write = int(sys.argv[1])
pidfile = sys.argv[2]
filename = sys.argv[3]

fd = os.open(filename, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
pidfd = os.open(pidfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
with os.fdopen(pidfd, 'w') as pidfobj:
    pidfobj.write('%s\n' % os.getpid())
with os.fdopen(fd, 'w') as f:
    os.close(status_pipe_write)
    for i in itertools.count():
        f.write('%s\n' % i)
        f.flush()
        time.sleep(1)

Now we have a daemon we can start in either a Type=forking systemd service unit or SysV initscript, and our launcher would be reusable for other kinds of payload with a bit of extra work.

Shedding most of the code and using systemd notify

If we accepted the limitation of only working with systemd, we could do a Type=notify systemd unit, by changing our notification mechanism, and do away with the launcher script.

#!/usr/bin/python
#payload-notify.py
import sys, os, time, itertools
from systemd.daemon import notify
pidfile = sys.argv[1]
filename = sys.argv[2]

fd = os.open(filename, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
pidfd = os.open(pidfile, os.O_WRONLY|os.O_NOCTTY|os.O_CREAT)
with os.fdopen(pidfd, 'w') as pidfobj:
    pidfobj.write('%s\n' % os.getpid())
with os.fdopen(fd, 'w') as f:
    notify('READY=1')
    for i in itertools.count():
        f.write('%s\n' % i)
        f.flush()
        time.sleep(1)