Fundamentals

Computer system organization

CPU, memory hierarchy, buses, and how the OS maps onto them

Start here

This page connects hardware reality (caches, buses, NUMA, interrupts) to why kernels are built the way they are. If terms like “cache line” or “DMA” are new, read the definition block next, it gives a minimal mental model before numbers and tables.

Von Neumann stored-program machine (theory)

Most general-purpose computers today follow a stored-program idea: instructions and data live in the same addressable memory. The CPU fetches instructions, decodes them, executes them, and may read or write memory operands. I/O devices (disks, NICs, keyboards) appear as special regions of addresses or dedicated instructions; the OS configures how they interact with RAM through DMA, interrupts, and MMU rules.

The OS does not change physics, it schedules work around latency hierarchies: registers and L1 caches are nanoseconds; DRAM is tens of nanoseconds; SSDs are microseconds; wide-area networks are milliseconds. Policies (NUMA placement, block I/O schedulers, IRQ affinity) exist because these gaps span orders of magnitude.

By the end of this page you should be able to

Name the main levels of the memory hierarchy and explain why each exists.
Explain why DMA uses physical addresses and why an IOMMU matters for security.
Relate interrupt storms and IRQ affinity to multi-core organization.
Read a simple latency table and infer likely bottlenecks for CPU-bound vs I/O-bound workloads.

Subsystems the OS maps onto

Each hardware subsystem exposes knobs the kernel must program.

CPU cores and caches

Cores execute instruction streams. Private L1/L2 (per core) and shared L3 reduce average memory latency. Cache coherence (e.g. MESI family) keeps copies consistent when multiple cores observe the same address, critical when locks or shared counters live in memory.

Main memory & NUMA

DRAM is organized in banks and channels. On NUMA servers, some RAM is “closer” to a socket; remote access costs extra latency. The OS allocator and scheduler try to keep threads near their memory.

Buses and peripherals

PCIe links NICs, GPUs, NVMe drives. Devices signal completion with interrupts; bulk data moves via DMA engines that read/write physical RAM while the CPU does other work.

Timers and preemption

A kernel needs periodic or high-resolution timer events to enforce scheduling quanta, profile workloads, and drive timeouts, without timers, a misbehaving process could monopolize the CPU in user mode forever on some architectures.

Formal reading of “organization”

Computer organization (this topic) is about structure and timing of the machine. Computer architecture sometimes emphasizes ISA design; operating systems sit above both and assume them as constraints. When you tune a database, you are often really tuning how much L3 miss traffic and how many remote NUMA accesses you generate.

TL;DR

The OS assumes a von Neumann-style machine: CPUs execute instructions from memory, caches hide DRAM latency, I/O devices appear as memory-mapped or port-I/O, and the OS configures timers, interrupt controllers, and the MMU to regain control periodically and enforce isolation.

Computer organization is the substrate operating systems exploit. Understanding caches, buses, and interrupt delivery explains why certain OS structures exist (per-CPU runqueues, RCU, slab coloring) and why naive locking or memory allocation can destroy performance even when “algorithmically optimal.”

Latency ladder (orders of magnitude)

Rough anchors, exact values vary by platform. Use them for intuition, not benchmarking.

Compare L1 → DRAM → SSD → WAN to see why batching and locality dominate systems work.

Latency orders of magnitude (rough, system-dependent)

Layer	Typical latency
L1 cache hit	~1 ns
Main memory (local NUMA node)	~50–100 ns
SSD read (4 KiB)	~10–100 μs
Cross-region network RTT	10–150 ms

Where latency hides

Symptoms vs layers, useful when debugging “slow but low CPU”.

Where latency hides

Layer	Typical culprit
CPU	Branch misses, pipeline hazards
Cache	Capacity/conflict misses, coherence traffic
DRAM	Row buffer misses, NUMA remote
I/O	Queue depth, rotational/FTL latency

Why the OS cares

Bus widths, cache hierarchy, and NUMA topology are not trivia, they change lock contention, DMA coherency costs, and why a “fast” algorithm becomes slow under false sharing. The OS schedules around these physical facts with affinity, per-CPU runqueues, and NUMA-aware allocators.

Hardware story (timeline)

Top-down: CPU execution down to block storage.

CPU executes streams of instructions

Pipeline stalls, branch prediction, and cache misses dominate latency more than raw GHz.

Memory hierarchy hides DRAM latency

L1/L2/L3 caches coherency protocols (MESI family) interact with kernel spinlocks and RCU.

Devices signal completion

MMIO registers, MSI-X interrupts, and DMA rings move data without byte-wise CPU copies.

Storage is block-oriented

OS stacks page cache atop block devices; alignment and queue depth matter for throughput.

Insights

Memory hierarchy is policy-driven

The hardware provides caches; the OS decides page placement, huge pages, and NUMA policies that determine whether workloads stay hot in local caches.

Interrupt controllers are programmable

APIC/IOAPIC/MSI-X route device events to specific CPUs, imbalance causes “IRQ storms” on one core unless the OS spreads affinity.

Timers drive preemption

The scheduler relies on periodic timer ticks or tickless high-resolution timers to enforce quanta and sample runqueues.

Firmware hands off a memory map

UEFI/ACPI tables tell the kernel which RAM is usable, reserved for devices, or unreliable, incorrect parsing is a classic boot bug class.

First-time debugging hint

When CPU utilization looks low but latency is high, think I/O wait, lock contention, or remote NUMA before assuming you need a “faster algorithm.”

Did you know?

Cache coherence protocols (MESI and variants) matter when kernel spinlocks share cache lines across cores, false sharing can dominate scalability.
DMA engines see **physical** addresses; IOMMUs translate so malicious devices cannot DMA arbitrary RAM.
Modern servers are NUMA: the OS must allocate memory “near” the scheduling CPU or pay remote DRAM latency.

Common mistakes

Assuming uniform memory

Remote NUMA access can double latency; `numactl` and default `first_touch` policy matter for big data jobs.

Ignoring cache line contention

Two cores writing different variables sharing a 64B line ping-pong cache, use padding or per-CPU counters.

Terms

NUMA

Non-uniform memory access, latency depends on which socket owns the DRAM.

MMIO

Memory-mapped I/O, device registers appear as physical addresses in the CPU space.

PCIe

Packetized peripheral bus; DMA engines and MSI-X live here.

Follow-ups

Why does false sharing hurt even when threads mutate different logical variables?
How does a device interrupt differ from a syscall with respect to preemption?
What breaks if DMA targets physical RAM a VM does not own?

Performance debugging

When CPU usage looks low but latency is high, suspect I/O wait, lock contention, or remote NUMA memory, not always “bad algorithms.”

Practice

Whiteboard exercise

Draw CPU → L1/L2/L3 → DRAM → NVMe. Annotate where TLB walk, page fault, DMA, and interrupt delivery occur.

Deep dive

Memory-mapped I/O vs port I/O

x86 historically had both; many devices appear as MMIO ranges the kernel maps into kernel virtual address space. Drivers read/write device registers like volatile memory with proper barriers.

Why the OS cares about cache lines

NUMA topology (Linux)

Terminal

lscpu | grep -i numa
numactl --hardware

Self-check

1DMA primarily reduces…

2NUMA stands for…

3IOMMU protects against…