pages tagged filesystemsyakkinghttp://yakking.branchable.com/tags/filesystems/yakkingikiwiki2014-03-04T09:32:28ZFile systemshttp://yakking.branchable.com/posts/filesystems/Richard Maw2014-03-04T09:32:28Z2014-02-05T12:00:09Z
<p>File systems are your interface to store your data. Modern file systems
offer a hierarchical view of your data, though historical file systems
have been flat.</p>
<p>However, your computer's hardware just knows about blocks of data. Your
operating system is responsible for translating the nice, human friendly
hierarchy into blocks of data that is written to a disk.</p>
<p>On Linux, you have the advantage of being able to choose from a variety
of file systems, suitable for different workloads. When installing your
operating system you may be given the opportunity to pick one. It's
important to know what you're choosing, so I'm going to describe a bit
of terminology, then descibe some common options.</p>
<h1>Terminology</h1>
<h2>Journalling</h2>
<p>In the case of a crash, it's helpful if you can determine what state
your filesystem is in. One approach for this is to have a journal,
where you write your changes to that before the rest of the storage,
and in the case of failure, the journal can be re-played to get the file
system into a consistent state.</p>
<h2>Copy-on-write</h2>
<p>Copy on write file systems, when re-writing new data to a file, instead
of overwriting the old data, create a copy of the data, write to that,
then update the original file pointers to point at the new data.</p>
<p>This allows similar atomicity guarantees to journalling, since either
the pointers point to the valid new data, or they point to the old data.</p>
<h2>fsck vs scrub</h2>
<p><code>fsck</code> stands for "File System ChecK". This is a step that can be
performed on a file system before it is being actively used to check
its integrity and fix errors.</p>
<p>This has the down-side that the file system can't be used while it's being
checked, so for routine maintanence the file system must be un-mounted.</p>
<p>Some file systems instead offer a "scrub" operation, which can be
performed while the file system is being used, and offers the same
functionality.</p>
<h1>File systems</h1>
<h2>FAT</h2>
<p>FAT stands for File Allocation Table, it is a relatively old file system,
files are limited to 4GiB in size and file names are case insensitive.</p>
<p>It has the benefit of being widely portable, being available on many
platforms. For this reason storage devices are often pre-formatted as
FAT, just so less technical users don't assume the device is broken and
return it.</p>
<p>Its portability makes it useful for USB flash drives and the partition
you use for <code>/boot</code>.</p>
<p>It would be a poor choice for your root file system.</p>
<h2>NTFS</h2>
<p>This is Microsoft's primary file system. Its data structures don't limit
the maximum file size to 4GiB.</p>
<p>As a commenter pointed out, NTFS is case sensitive, but not through the
Windows API, which maintains case insensitivity for compatibility with
older software that assumed insensitivty, since it was dealing with FAT.</p>
<p>On Windows it is case preserving, so if you create a file called "Foo",
you can read it as "foo", but when you list the contents of the directory,
it is shown as "Foo", rather than "FOO", as FAT traditionally does.</p>
<p>This makes it a better choice for storage media, as Linux is also able
to read it.</p>
<p>It is still inadvisable to use NTFS as your root file system on Linux,
since it's primary use is reading disks that are also used by Windows
machines, rather than being an installation's root file system.</p>
<h2>ext2, ext3 and ext4</h2>
<p>The ext file systems are Linux's primary file systems and are usually
the default option when installing Linux distributions.</p>
<p>Despite having a similar name, they are different beasts. ext2 is rather
primitive, only really useful with old bootloaders.</p>
<p>ext3 is more advanced, though active development has moved on to ext4.</p>
<p>ext4 supports journalling, uses extents for its storage and supports
extended attributes, where additional metadata can be assigned to a file.</p>
<p>There are third-party tools to read ext file systems from Windows,
but NTFS support in Linux is better than ext support in Windows.</p>
<h2>XFS</h2>
<p>XFS is a development from Silicon Graphics. It exceeds ext4's features,
including the ability to take snapshots of the logical state of the file
system, and was the source of extended attributes.</p>
<p>It is available on IRIX and Linux, so portability is not its strong point,
hence would not be useful on a USB flash drive, but it is an excellent
choice for your root file system.</p>
<h2>ZFS</h2>
<p>ZFS is a product of Sun Microsystems, later bought by Oracle. It is a
very advanced file system, offering all the features mentioned above
plus more.</p>
<p>This is a copy-on-write file system, unlike the above, which were either
journalling, or wrote to the blocks directly.</p>
<p>Being copy-on-write allows for deduplication, since if multiple files
have the same data, the file system can point both files to the same
data, since if it's changed in one file, the other one won't have its
data changed.</p>
<p>Live file system checking is possible with the scrub command, so downtime
is not needed to perform maintanance.</p>
<p>It can use multiple physical devices as its storage pool.</p>
<p>Its license is incompatible with the Linux kernel, so kernel support is
provided by a third-party module, which makes it possible that a kernel
update could leave your file system unreadable, since the ZFS kernel
module is un-readable.</p>
<p>Loading an external kernel module is slower than it being built in,
so this impacts boot speed.</p>
<p>Despite its complexity, ZFS is also available on the Solaris and BSD
unices.</p>
<h2>BTRFS</h2>
<p>Work on BTRFS was initiated by Oracle to be a competitor to
ZFS, this is no longer the motivating factor, since Oracle acquired Sun,
but BTRFS is likely to become the default Linux file system in the future.</p>
<p>It is nearly as featureful as ZFS, only missing the online deduplication,
which the BTRFS developers expect to complete in a couple of Linux
kernel releases.</p>
<p>Its design allows for a transition between ext and btrfs with the
<code>btrfs-convert</code> tool, by saving the ext metadata elsewhere and re-mapping
ext's data to btrfs extents. It still offers the original file system's
data as a read-only disk image that can be mounted. Reverting back to
ext is done by reinstating the ext metadata.</p>
<p>Unfortunately, it has a reputation of being unstable, corrupting your
data and becoming unusable. This is a natural stage of file system
maturity though, and BTRFS is my preferred root file system.</p>