Explain Docker's overlay filesystem in detail. How do layers work during reads and writes? What is copy-on-write, and how does the overlay2 storage driver differ from AUFS and devicemapper?
Quick Answer
Overlay2 uses Linux overlayfs to stack read-only image layers (lowerdir) with a writable container layer (upperdir). Reads traverse layers top-down until the file is found. Writes trigger copy-on-write: the file is copied from the lower layer to the upper layer before modification. Overlay2 is superior to AUFS (not in mainline kernel) and devicemapper (block-level, poor performance for many files).
Detailed Answer
Overlay Filesystem Architecture
Linux overlayfs merges multiple directory trees into a single unified view. Docker uses this to combine image layers (immutable) with a container layer (writable).
Key Directories
- lowerdir: Read-only image layers, stacked in order (multiple layers separated by colons) - upperdir: Writable container layer where all modifications go - workdir: Temporary workspace for atomic operations (rename, copy-up) - merged: The unified view presented to the container process
Read Operation
When a container reads a file, overlayfs searches from top (upperdir) to bottom (lowerdirs). The first layer containing the file is returned. This is O(1) for files in the upperdir, O(n) for files only in deep lower layers (but kernel optimizes with dcache).
Write Operation (Copy-on-Write)
When a container modifies a file that exists in a lower layer: 1. The entire file is copied from the lower layer to the upperdir (copy-up) 2. The modification is applied to the copy in upperdir 3. Subsequent reads see the modified version in upperdir This means the first write to a large file is expensive (must copy entire file), but subsequent writes are fast.
Delete Operation (Whiteout)
Deleting a file from a lower layer creates a 'whiteout' character device in the upperdir. The file appears deleted in the merged view but still exists in the lower layer (image layer is immutable). Deleting a directory creates an 'opaque whiteout'.
Storage Driver Comparison
- overlay2 (current default): Uses kernel overlayfs, supports up to 128 lower layers, efficient metadata handling, best overall performance - AUFS: Similar union filesystem but never merged into mainline Linux kernel (only Ubuntu/Debian patches). Deprecated. - devicemapper: Block-level CoW using LVM thin provisioning. Each layer is a block device snapshot. Heavy overhead for file-level operations, complex setup (requires direct-lvm for production). Deprecated. - btrfs/zfs: Filesystem-level snapshots. Good for specific use cases but not general purpose Docker.