Filesystem Internals for System Design

System design interviews care about filesystems when estimating disk behavior, arguing for/against storing many tiny files, explaining fsync latency, or justifying why a storage engine uses direct I/O. The mental model is layered: application → VFS → concrete FS (ext4, XFS, NTFS, APFS) → block device → media.

Crash consistency is the recurring theme. Metadata structures (bitmaps, inode tables, directory trees) must not be half-updated. Journals or copy-on-write schemes make recovery bounded. Data durability is a separate promise you opt into with barriers.

An inode does not contain the path. Multiple directory entries (hard links) can reference one inode. Symlinks are a different inode type storing a path string. Device nodes, sockets, and FIFOs are also inode types—useful when you see “files” that are not byte streams.

Classic Unix used direct, single-, double-, and triple-indirect block pointers. Large files paid many metadata reads. Modern extent maps describe contiguous runs compactly, which matches how chunkservers and log files want to stream.

inode (conceptual):
  mode, uid, gid, size, timestamps, nlink
  extent_tree: [(phys_block=1000, len=8), (phys=5000, len=4), ...]

directory "logs":
  ("app.log", inode=42)
  ("app.log.1", inode=43)

unlink("app.log"):
  remove dentry; nlink--
  if nlink==0 and no open fds: schedule free of blocks + inode

Internal fragmentation:
  1 byte file still occupies ≥ 1 block (plus inode)
  → millions of tiny files waste space and inode table slots

The inode cache and dentry cache are why “create 10 million files in one directory” becomes a control-plane nightmare for local disks and for DFS namenodes alike. Shard directories, use packed containers (tar, SSTables, object layouts), or a dedicated metadata store when cardinality explodes.

A successful write() usually updates the page cache. Dirty pages flush later via writeback. fsync forces the durable point the application means by “committed to disk.” Renaming a file over an old one is a common atomic publish pattern—but only if the data was fsynced before the rename, and directory entries are made durable as required on that OS.

Safe-ish replace pattern (simplified):
  fd = open("file.tmp")
  write(fd, data)
  fsync(fd)
  close(fd)
  rename("file.tmp", "file")   // atomic namespace update on same FS
  fsync(directory_fd)          // on some systems, required for crash safety

Without fsync:
  power loss → empty/truncated file or old bytes
  yet the app may have already ACKed the user

Storage engines that need predictable control use O_DIRECT to bypass the page cache, manage their own block cache, and issue explicit flushes. That avoids double caching and surprise writeback storms—at the cost of reimplementing caching wisely.

Copy-on-write filesystems (btrfs, ZFS-style) take a different approach: write new blocks, then atomically swing root pointers. Checksums and snapshots fall out naturally; fragmentation and free-space management become the hard parts. For interviews, say “journal vs COW” as two families of crash consistency, then pick based on workload.

Delayed allocation lets the FS wait until writeback to pick physical blocks, improving contiguity for large streaming writes.fragmentation on HDDs shows up as collapsed sequential throughput. On SSDs, random vs sequential matters less for seeks but write amplification, queue, and queue matter more—especially if the FS and the device both garbage-collect.

Reads through mmap still use the page cache. Writes dirty pages that must be msync'd (or equivalent) for durability—the same fsync lesson under another name. Concurrent writers need explicit synchronization; the filesystem does not serialize your store's logical transactions just because two processes mapped the same path.

Locking belongs in the application or engine: file locks (flock, POSIX locks) are advisory on many systems and do not substitute for a WAL protocol. Distributed systems that share a network filesystem (NFS) inherit weaker cache-coherency assumptions—another reason chunkservers prefer local disks.

When mmap helps in SD contexts:
  + read-only SSTable / index files with warm page cache
  + shared read-mostly lookup tables in one process tree

When to avoid as primary write path:
  - unclear msync barriers vs crash windows
  - multi-writer mutation without an engine-level log
  - remote FS mounts with surprising consistency