System Design Foundations

Operating System Basics for System Design

Load balancers and Kafka diagrams still run on kernels. When a Node service hits EMFILE, a JVM spends seconds in GC, or Postgres waits on , the outage is an operating-system story wearing a distributed-systems costume.

This note covers the OS ideas that show up in interviews and production: processes vs threads vs , , , scheduling, durability, and container limits—always as design constraints.

OS → design link

Concurrency model

Threads, processes, or event loop—pick for the workload.

Hard caps that autoscaling can smash.

Page faults and OOM beat clever cache math.

Durability

fsync and write barriers set write QPS ceilings.

What actually happens

A request arrives on a socket (). The kernel wakes a or an event loop, your code runs a bit of CPU, then waits again on disk or network. decides when you run; and cache coldness decide how much of the CPU you waste when you oversubscribe.

Memory is : the working set must fit in RAM or you pay . Containers wrap this with so “host has RAM” does not mean “your pod may use it.” Durability goes through page cache until (or equivalent) pushes bits to media. Connection pools are just userspace discipline around kernel sockets and server-side .

At scale, missing turns a slow dependency into an OOM. OS basics are how you notice that early.

How to choose a concurrency model

1. Mostly waiting on I/O? Prefer or a modest thread pool with non-blocking clients. Huge thread-per-connection designs spend their life switching.

2. Need isolation? Separate (or containers) limit blast radius for crashes and memory corruption at higher overhead.

3. Heavy CPU inside the request? Offload to worker pools or separate services so one encode job cannot stall thousands of light handlers on a single event loop.

Simple rule of thumb

Match the concurrency model to the wait. Size thread/connection pools against the downstream hard limit. Treat cgroup memory as the real ceiling. Put fsync on paths that must not lie about durability.

Request path through the OS

mermaid

flowchart TB
    NIC[NIC / interrupt / NAPI] --> SOCK[Socket receive queue]
    SOCK --> ACC[Accept / epoll readiness]
    ACC --> APP[App thread or event loop]
    APP --> CPU[User CPU: parse / auth / logic]
    APP --> NET2[Outbound socket to DB/cache]
    APP --> DISK[Read/write file or WAL]
    DISK --> PC[Page cache]
    PC --> FSYNC[fsync / device barrier]
    FSYNC --> MEDIA[SSD/HDD]
    APP --> RESP[Response bytes to socket]

Interview usefulness: when p99 rises but CPU looks “only 40%,” ask about run-queue latency, disk await, lock contention, GC, and downstream pools—not only business logic complexity.

Processes, threads, and async

Model	Strength	Failure mode
Process per worker	Isolation, simple mental model	Memory duplication; slower spawn
Thread pool	Familiar blocking code	Oversubscription → context-switch storm
Async event loop	Huge concurrency per core	CPU work blocks everyone on that loop
Hybrid	Async I/O + CPU worker pool	More moving parts; queue sizing matters

plaintext

Rough capacity intuition (I/O-bound API):

  Blocking thread-per-request:
    threads ≈ concurrent in-flight requests
    10k concurrent waits → 10k threads (usually disastrous)

  Async loop:
    10k sockets, few threads
    Still need a cap: memory per connection + downstream pools

  Rule:
    concurrent_in_flight ≈ arrival_rate × latency
    Size pools from that Little's-Law style estimate, then add headroom.

File descriptors and connection math

mermaid

sequenceDiagram
    participant LB as Load balancer
    participant A as App replicas (N)
    participant DB as Database

    LB->>A: Traffic scales N upward
    A->>DB: Each replica opens pool size P
    Note over DB: Connections ≈ N × P
    DB-->>A: max_connections exceeded
    A-->>LB: Errors / timeouts
    Note over LB,DB: Fix: lower P, PgBouncer-style pooling, or scale DB capacity first

Each TCP connection consumes a on client and server. OS ulimit -n, systemd limits, and database max_connections are hard walls. Soft walls include memory per connection and CPU to handshake TLS.

Design move: prefer multiplexed protocols and proxy poolers in front of databases; set explicit pool sizes; load-test reconnect storms after deploys.

Virtual memory, page cache, and OOM

RAM is shared by your heap, native allocations, and the OS page cache (which makes disk look fast until it does not). Under pressure, the kernel reclaims cache and may swap; major create multi-millisecond stalls that show up as p99 cliffs.

plaintext

Memory pressure timeline:

  T0  Working set fits; page cache hot; p99 calm
  T1  New feature doubles heap; cache shrinks
  T2  Disk reads rise; CPU iowait climbs
  T3  cgroup limit hit → OOMKill of the pod
  T4  Restart stampede reconnects pools; neighbors suffer

Design habits:
  - Size heaps against container limits, not node size
  - Leave room for page cache on DB hosts
  - Prefer shedding load to swapping on latency paths
  - Watch container memory working set, not only host free RAM

Scheduling, noisy neighbors, and headroom

latency—the wait between “runnable” and “running”—matters for tail latency. On multi-tenant hosts or dense containers, a “CPU throttle” from a can look like application slowness. Blocking servers with too many threads increase and make the problem worse.

Signals something is OS-bound

High run-queue / load average with modest “app CPU”
CPU throttling metrics in the container
Disk await spikes correlating with p99
Thread counts climbing without QPS gains

Typical fixes

Right-size CPU limits and requests
Cap thread pools; move to async where appropriate
Isolate noisy batch jobs from latency pods
Keep utilization headroom for bursts

Durability: page cache vs fsync

Group commit batches many transactions into one sync to raise throughput. Asynchronous durability acknowledges earlier and accepts loss windows—fine for metrics, dangerous for ledgers. Replication is not a substitute for local durability unless you carefully define failure domains (power loss of one rack vs correlated loss).

Design interview line: “For payments we need durable commit before ack; write QPS is therefore gated by sync latency and group commit. For feed fan-out we acknowledge after queue persist with replication.”

Backpressure beats OOM

Listen queues, bounded in-memory channels, HTTP 503 with Retry-After, and load shedders are how applications cooperate with the OS instead of fighting it. Unlimited buffering converts a slow dependency into memory death.

plaintext

Without backpressure:
  accept → unbounded queue → workers → slow DB
  queue grows → RSS climbs → OOMKill → restart → repeat

With backpressure:
  accept with limited listen backlog
  bounded queue; shed or fail fast when full
  timeouts free stuck workers
  clients retry with jitter / backoff
  system stays within cgroup memory

Trade-offs

Choice	Gain	Cost
Thread-per-request	Simple blocking code	Poor scaling; switch overhead
Async everywhere	High concurrency density	Complexity; CPU hazards on the loop
Large connection pools	Absorb bursts	Overwhelm DB; fd pressure
Strict fsync	Crash-safe acks	Lower write throughput
Tight cgroup limits	Density, noisy-neighbor control	Throttle/OOM under real peaks
Unlimited buffering	Smooths short blips	Turns outages into OOMs

Worked example: “scale the API” outage

Product doubles replicas from 20 → 40 during a launch. Each replica holds a DB pool of 50. Database max_connections=1000.

plaintext

Before: 20 × 50 = 1000 connections (at the wall already)
After:  40 × 50 = 2000 attempted connections

Symptoms:
  - new replicas cannot get DB connections
  - health checks flap
  - old replicas also fail as pools churn
  - on-call “fixes” by raising max_connections → DB memory dies

OS/design fix:
  - drop pool size to 20 (40 × 20 = 800)
  - add a pooler proxy
  - gate autoscaling on connection budget, not only CPU
  - load-test deploy/restart reconnect storms

Choosing and explaining OS basics in system design

A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.

Explain the core idea

Tie concurrency model to whether work is I/O-bound or CPU-bound.

Do the connection and FD math across replicas before claiming horizontal scale.

Call out memory cgroup limits, page-fault risk, fsync durability, and backpressure explicitly.

When it belongs

Sizing thread pools, event loops, or database connections.

Discussing durability of writes or crash behavior.

Explaining mysterious latency under memory or CPU throttle.

How it fails

FD or max_connections exhaustion after scale-out.

Event-loop blocking from CPU-heavy work.

OOM from unbounded queues without backpressure.

Page-fault storms when the working set exceeds RAM/cgroup limits.

Trade-offs to weigh

Isolation (processes/containers) versus memory and startup cost.

Durability (fsync) versus write throughput.

Pool size headroom versus downstream overload risk.

Common misunderstandings

Treating OS topics as unrelated to distributed design.

Autoscaling replicas without a connection budget.

Assuming write() equals durable.

Using host-level free memory as the container’s real limit.

Wrap-up

Operating systems turn diagrams into running services. Concurrency, descriptors, memory, scheduling, and durability are first-class design inputs—especially once containers and autoscaling enter the picture.

Exercises

Size a pool. 500 QPS, 40 ms mean DB latency, 30 app replicas. Using Little's Law, estimate concurrent DB waits and propose a per-replica pool size with headroom under a 900 connection budget.
Pick a concurrency model for an image-thumbnail service that both accepts uploads and runs CPU encode. Sketch the loop vs worker split.
Durability choice. For “like count” vs “wallet debit,” say whether you need fsync-before-ack and what you tell the user on crash mid-write.

Key takeaway

System design sits on OS reality: how you wait, how many fds you open, whether memory fits, whether writes are durable, and whether you apply backpressure before the OOM killer does it for you.