Locking is a concurrency primitive, usually allowing shared/read locking and exclusive/write locking.

Linux allows you to take locks on files.

There's the POSIX lockf(3) file locks, which let you lock a specific range of a file, but have various pitfalls.

There's BSD flock(2) file locks, which have per-file-descriptor rather than per-process locks, but don't allow locking a file range.

There's also the new (since Linux 3.15) Linux-specific F_OFD_{SETLK,SETLKW,GETLK} fcntl(2) commands, which are file-descriptor bound and offer file ranges.

I'm only interested in the file-descriptor bound locks, and to keep that simple I'm not going to use file ranges, so we're going to discuss flock(2) and flock(1).

For the sake of conciseness, I will be using python and shell for examples, rather than using the C api directly.

Taking locks

The basic principle is that, since it's a file-descriptor bound lock, you need to open the file, then use the locking call on the file descriptor.

The following programs can be used to atomically generate unique IDs.

They will wait for any other programs that may be using the file to finish before reading, updating and writing to it.

#/bin/sh
LOCKFILE="$1"
shift
exec 100<>"$LOCKFILE" # open file descriptor for read-write
flock 100
read -u 100 num
printf '%d\n' "$num"
num="$(expr "$num" + 1)"
# Need to re-open to be able to write to the beginning
printf '%d\n' "$num" >/proc/self/fd/100

The logic is similar in python, but without having to shell out to flock(1).

#!/usr/bin/python
import sys
from fcntl import flock, LOCK_EX, LOCK_SH, LOCK_UN
lockfile = sys.argv[1]
with open(lockfile, 'rw') as f:
    flock(f.fileno(), LOCK_EX)
    num = int(f.read())
    print(num)
    num += 1
    f.seek(0)
    f.write('%d\n' % num)

Releasing locks

File descriptor locks are released when every reference to the file descriptor is closed, or explicitly with flock(fd, LOCK_UN).

Since file descriptors are closed on process termination, the shell program will release the lock when its process terminates.

The python program uses a context manager with the file, which means it will close the file at the end of the scope, so the file will be closed before termination.

If this were made more explicit, the shell program would end with:

flock -u 100
exec 100>&-

The python program would end with:

    flock(f.fileno(), LOCK_UN)

Using flock(1) with a continuation command hence closes the lock after the command has run.

Lock contention

The purpose of locking is to ensure that resources are protected while the lock is held.

If a lock is not held by any open file descriptors, you can always take it.

There are two ways to hold a lock, exclusively and shared.

The former is the default for the flock(1) command, though it can be explicitly chosen with the --exclusive option, or with the LOCK_EX flag as shown with the python program using the flock(2) syscall.

The latter can be requested with the --shared option, or the LOCK_SH flag.

If a lock is currently held with an exclusive lock, or you want to take an exclusive lock and it is already locked, you can't take the lock.

If the held lock is a shared lock though, you can take a shared lock on the file.

Blocking vs non-blocking

When you are contended on taking a lock, you can either wait for the lock to be released, or fail immediately so you can try something else.

When attempting to take a contended lock, by default you wait for it to be released, however when using flock(2) you can instead of passing LOCK_EX or LOCK_SH, pass LOCK_EX|LOCK_NB and LOCK_SH|LOCK_NB to make this a non-blocking lock, which will immediately return if the lock is contended.

When using flock(1) you would pass --nonblock to do this, and while blocking is the default, you can pass --wait to make it block explicitly.

Blocking locks have the advantage that your process will be suspended until you can take the lock, you are woken up as soon as you can take the lock, and if there is a queue of processes wanting to take the lock, then processes that are waiting get the lock before those that weren't.

However the danger of blocking locks, is that if the other lock doesn't get released, you will not be woken up.

This is a problem when your process needs to be responsive to input.

This can be worked around by having a separate thread to handle user responses, but at some point you've got to draw the line, and say that not being able to take the lock in time is an error.

The neatest way to do this is to use flock(1)'s --timeout option, which you would use from python as:

from subprocess import check_call, CalledSubprocessError
from errno import EAGAIN
from os import strerror
def take_lock(fd, timeout=None, shared=False):
    try:
        check_call(['flock',
                    '--wait' if timeout is None else '--timeout=%d' % timeout,
                    '--shared' if shared else '--exclusive',
                    '--conflict-exit-code=75', #EX_TEMPFAIL
                    str(fd)])
    except CalledSubprocessError as e:
        if e.returncode == 75:
            raise IOError(EAGAIN, strerror(EAGAIN))
        raise

with open(lockfile, 'r') as f:
    take_lock(f.fileno(), timeout=30, shared=True)

Note: Old versions of flock(1) may not support --conflict-exit-code.

It is possible to do locking with a timeout in native python code, by using setitimer(2),

#!/usr/bin/python
from fcntl import flock, LOCK_SH, LOCK_EX, LOCK_NB
from os import strerror
from signal import signal, SIGALRM, setitimer, ITIMER_REAL
from sys import exit

def take_lock(fd, timeout=None, shared=False):
    if timeout is None or timeout == 0:
        flock(fd, (LOCK_SH if shared else LOCK_EX)|(LOCK_NB if timeout == 0 else 0))
        return
    signal(SIGALRM, lambda *_: None)
    setitimer(ITIMER_REAL, timeout)
    # Racy: alarm could be delivered before we try to lock
    flock(fd, LOCK_SH if shared else LOCK_EX)

if __name__ == '__main__':
    from argparse import ArgumentParser
    parser = ArgumentParser()
    parser.add_argument('--shared', action='store_true', default=False)
    parser.add_argument('--exclusive', dest='shared', action='store_false')
    parser.add_argument('--timeout', default=None, type=int)
    parser.add_argument('--wait', dest='timeout', action='store_const', const=None)
    parser.add_argument('--nonblock', dest='timeout', action='store_const', const=0)
    parser.add_argument('file')
    parser.add_argument('argv', nargs='*')
    opts = parser.parse_args()
    if len(opts.argv) == 0:
        fd = int(opts.file)
        take_lock(fd, opts.timeout, opts.shared)
    else:
        from subprocess import call
        with open(opts.file, 'r') as f:
            take_lock(f.fileno(), opts.timeout, opts.shared)
            exit(call(opts.argv))

However, since signals are process-global state, doing it that way can result in a process that has interesting side-effects, especially in a threaded environment, which makes it harder to reason about the behaviour of the program.

It may be nicer to run the blocking flock(2) in a subprocess, just to avoid having to make your main program deal with signals.

The following version works as-before, but uses multiprocessing to run the flock(2) code in a subprocess, and pass any exceptions back to the main process.

#!/usr/bin/python
from errno import EINTR, EAGAIN
from fcntl import flock, LOCK_SH, LOCK_EX, LOCK_NB
from multiprocessing import Pipe, Process
from os import strerror
from signal import signal, SIGALRM, setitimer, ITIMER_REAL
from sys import exit

def _set_alarm_and_lock(fd, pipew, timeout, shared):
    try:
        signal(SIGALRM, lambda *_: None)
        setitimer(ITIMER_REAL, timeout)
        # Racy: alarm could be delivered before we try to lock
        flock(fd, LOCK_SH if shared else LOCK_EX)
    except BaseException as e:
        # This loses the traceback, but it's not pickleable anyway
        pipew.send(e)
        exit(1)
    else:
        pipew.send(None)
        exit(0)

def take_lock(fd, timeout=None, shared=False):
    if timeout is None or timeout == 0:
        flock(fd, (LOCK_SH if shared else LOCK_EX)|(LOCK_NB if timeout == 0 else 0))
        return
    piper, pipew = Pipe(duplex=False)
    p = Process(target=_set_alarm_and_lock,
                args=(fd, pipew, timeout, shared))
    p.start()
    err = piper.recv()
    p.join()
    if err:
        if isinstance(err, IOError) and err.errno == EINTR:
            raise IOError(EAGAIN, strerror(EAGAIN))
        raise err

if __name__ == '__main__':
    from argparse import ArgumentParser
    parser = ArgumentParser()
    parser.add_argument('--shared', action='store_true', default=False)
    parser.add_argument('--exclusive', dest='shared', action='store_false')
    parser.add_argument('--timeout', default=None, type=int)
    parser.add_argument('--wait', dest='timeout', action='store_const', const=None)
    parser.add_argument('--nonblock', dest='timeout', action='store_const', const=0)
    parser.add_argument('file')
    parser.add_argument('argv', nargs='*')
    opts = parser.parse_args()
    if len(opts.argv) == 0:
        fd = int(opts.file)
        take_lock(fd, opts.timeout, opts.shared)
    else:
        from subprocess import call
        with open(opts.file, 'r') as f:
            take_lock(f.fileno(), opts.timeout, opts.shared)
            exit(call(opts.argv))

Converting locks

flock(2) with LOCK_SH when you have a LOCK_EX, or --shared with --exclusive when using flock(1), turns it from an exclusive lock to a shared lock.

Similarly, you can go the other way, converting a shared lock into an exclusive one, though this counts as a contended lock if there are any other holders of shared locks.

You may want to do this if it's for managing the lifetime of a resource.

You would want to hold an exclusive lock on it when making the resource, so that any concurrent users can know that it's being set up, so they can take a blocking lock and wait for it to be ready.

After the resource has been set up, you would convert it to a shared lock, so you can use it yourself, and any other processes wanting to take a shared lock to use it can be woken up and start using it.

When you are finished using the resource you can convert it to an exclusive lock.

You will then know that when you have taken the exclusive lock, that there can be no other users of the resource, so it is safe to remove it.

Lock conversion is important, as you don't want to unlock then re-take the lock, as there is a period where it is unlocked, which other concurrent users might decide means it can be cleaned up.

You'd end up cleaning it up just after setting it up, before you had a chance to use it.