System Design Infrastructure
Virtualization Technologies
A cloud provider sells you "four vCPUs and 16 GB." Underneath, dozens of tenants share sockets, DRAM, and NICs. Without isolation, one noisy neighbor—or one escape—ruins everyone. Virtualization is how hardware becomes a safely multiplexed product.
This note contrasts and VMs with , explains why matter, and when or device passthrough change the performance story.
Pick an isolation unit
Need a kernel boundary?
Multi-tenant or untrusted → VM / microVM.
Trusted workloads → containers on shared kernel.
Virtio or SR-IOV; avoid naive emulation.
cgroups / VM sizes—design to the limit.
What actually happens
Bare metal gives one OS the whole machine. That is simple and lonely: utilization is low, provisioning is slow, and tenants cannot safely share. The naive virtualization of the past—pure software emulation—was too slow for servers. Hardware assists (Intel VT-x / AMD-V) plus modern hypervisors made multi-tenant clouds practical by trapping sensitive operations with tolerable overhead.
A presents virtual CPUs and memory maps to each guest. When the guest executes something sensitive, the CPU takes a ; the hypervisor emulates or mediates, then resumes. Device models matter enormously: emulated NICs exit constantly; with virtio uses shared rings and fewer exits; SR-IOV passthrough (with an ) approaches bare-metal networking at the cost of flexibility.
Containers take a different cut. They share one host kernel and isolate with and . Startup is milliseconds; packing is dense; the blast radius of a kernel bug is the host. Clouds often nest both: containers inside VMs so each tenant still has a boundary.
Overcommit and reclaim memory under pressure. lets a VM host further VMs for labs or nested products—with multiplied exit costs. System design interviews care about which boundary you chose and what overhead or failure mode you accepted.
How to choose VM vs container
1. How strong must the boundary be? Untrusted multi-tenant code, hostile customers, or regulated isolation → VM or microVM. Same-team services → containers usually suffice.
2. How fast must cold start be? Functions and scale-to-zero love containers or specialized microVMs. Full guest boots cost seconds unless pre-warmed.
3. Who owns the kernel ABI? Containers inherit the host kernel. If you need an older kernel module or a custom network stack, a VM (or VM-per-tenant) is the honest answer.
Simple rule of thumb
Default to containers for app packing on trusted hosts; put a VM boundary around each trust domain; use virtio (or passthrough with IOMMU) for I/O-heavy paths; always size against the virtual limit, not the physical host.
Isolation stack
Hardware assists and the IOMMU sit under both VMs and container hosts. Choose where the trust boundary cuts the stack.
Read the diagram definition
flowchart TB
subgraph hw [Hardware]
CPU[CPU + VT-x/AMD-V]
MEM[DRAM]
NIC[NIC / NVMe]
IOMMU[IOMMU]
end
subgraph hyp [Hypervisor / host]
HV[Hypervisor or host kernel]
VMM[VMM / QEMU]
end
subgraph guests [Isolation units]
VM1[VM: guest kernel + apps]
CTR[Containers: namespaces + cgroups]
end
CPU --> HV
MEM --> HV
NIC --> IOMMU --> HV
HV --> VMM --> VM1
HV --> CTR
Hypervisors: Type-1, Type-2, and exits
hypervisors minimize the trusted computing base on the metal and dominate public cloud. KVM blurs the taxonomy: the Linux kernel hosts guests while QEMU/VMM provides devices—operationally treated as bare-metal-class. tools are fine for developer VMs; they are rarely the isolation story you want for hostile tenants.
Performance intuition is exit math. Every exit costs context and cache disruption. Idle guests still pay for timer ticks and interrupts unless carefully tuned. I/O-bound guests without virtio waste cycles in emulation. CPU-bound guests with good scheduling can approach native if exits are rare.
Read the diagram definition
sequenceDiagram
participant G as Guest vCPU
participant H as Hypervisor
participant D as Emulated / virtio device
G->>G: Privileged or I/O instruction
G->>H: VM exit
H->>D: Emulate or kick virtio ring
D-->>H: Completion
H->>G: VM entry / resume
Note over G,H: Fewer exits → closer to bare metal
| Approach | Idea | Cost / caveat |
|---|---|---|
| Full emulation | Hypervisor fakes legacy devices | Many exits; compatibility high |
| Paravirt / virtio | Guest drivers use shared rings | Needs drivers; cloud default |
| Passthrough / SR-IOV | Guest owns a VF with IOMMU | Near-native; less live-migrate flexibility |
Containers as OS-level virtualization
A is not a tiny VM. It is a process tree with a private mount/network/PID view () and resource caps (). Images ship userspace filesystems; syscalls hit the host kernel. That is why "runs on my laptop Docker" can fail on a host with a different kernel feature set—and why container escapes that reach the kernel are host-wide incidents.
Seccomp, AppArmor/SELinux, read-only roots, and dropped capabilities harden containers but do not equal a guest kernel. gVisor, Kata, and Firecracker-style microVMs sit between: they restore stronger isolation with more overhead than plain runc.
Orchestrators schedule containers; they do not invent isolation. If two containers share a kernel, your threat model must accept that. Many regulated designs therefore schedule containers inside per-tenant VMs—defense in depth with a density penalty.
Memory, devices, and nesting
Memory overcommit
Hosts often sell more guest RAM than DRAM. and host swapping reclaim under pressure. Guests see mysterious latency; design SLOs assuming noisy memory, or buy dedicated hosts.
Device passthrough
GPUs and NICs via VFIO/ give near-native speed. Live migration and oversubscription get harder. Match passthrough to workloads that need it.
Nested virtualization
enables nested hypervisors and some managed products. Expect higher exit costs and feature gaps; measure before promising nested KVM performance.
MicroVMs
Slim VMM + minimal devices (Firecracker-style) cut boot time and attack surface for functions and multi-tenant sandboxes—VM isolation with container-like operational goals.
Comparison cheat sheet
| Property | VM | Container |
|---|---|---|
| Kernel | Own guest kernel | Shared host kernel |
| Cold start | Seconds (unless microVM/warm) | Milliseconds–subsecond |
| Density | Lower (full OS memory) | Higher |
| Isolation | Stronger hardware + kernel boundary | Process-level; kernel shared fate |
| I/O path | Virtio / passthrough | Host syscall / host networking |
Numbers that shape intuition
| Metric | Order of magnitude | Design note |
|---|---|---|
| VM exit cost | ~1–10+ µs class each | Chatty emulated I/O multiplies exits |
| Container start | ~10–100 ms typical | Image pull dominates cold path |
| Classic guest boot | seconds–tens of seconds | Pre-warm pools for scale-out |
| MicroVM boot | ~100 ms class (tuned) | Enables stronger sandbox per request |
Failure timeline: noisy neighbor
T0 Two tenants share a host; both on virtio NICs; memory overcommitted T1 Tenant A batch job allocates; ballooning deflates Tenant B T2 B starts guest swapping; checkout p99 jumps while CPU looks fine T3 A's emulated disk path (no virtio) burns exits; steal time rises on B T4 Fix: dedicated hosts / larger reservations, virtio everywhere, alert on steal + balloon Outcome Virtualization multiplexes well until silent overcommit becomes user pain
Break the improved design
Dedicated hosts still fail if the guest kernel OOMs because the app ignored cgroup/VM limits. Virtio helps until a buggy guest driver or host dataplane regresses. MicroVMs reduce attack surface but do not remove the need for image hygiene and syscall policy.
Nested virtualization for "Kubernetes in a VM in a VM" can look elegant on a slide and catastrophic on a latency budget. Measure exit rates and steal time; do not inherit marketing density numbers.
Trade-offs
Isolation strength vs density
VMs and microVMs cost memory and boot time for stronger boundaries. Containers maximize density and share kernel fate. Put the expensive boundary at the trust domain, then pack containers inside.
Compatibility vs exit reduction
Emulated devices boot anything and waste CPU. Paravirt needs drivers and wins on throughput. Passthrough wins further and complicates migration. Prefer virtio unless you have a measured reason not to.
Overcommit savings vs tail latency
Memory and CPU overcommit raise utilization until ballooning and steal time burn p99. Latency-critical products often pay for reserved capacity; batch can tolerate denser packing.
Passthrough performance vs operability
SR-IOV and GPU passthrough need IOMMU, pinning, and careful capacity planning. Live migrate and oversubscribe become harder. Use when the workload pays for near-native I/O.
Nested convenience vs multiplied exits
Nesting enables test clouds and some product shapes. Each layer adds exits and scheduling opacity. Prefer flat topologies for production latency paths.
Choosing and explaining virtualization
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
VMs multiplex hardware with a guest kernel boundary; containers multiplex one kernel with namespaces and cgroups.
Performance is mostly about reducing VM exits and choosing virtio or passthrough wisely.
I place the strong isolation boundary at the trust domain and pack density inside it.
When it belongs
Multi-tenant packing, cloud instance types, or container vs VM debates appear.
The interviewer asks about noisy neighbors, steal time, or cold start.
You need to justify microVMs or passthrough GPUs.
How it fails
Noisy neighbors via overcommit, ballooning, and steal time.
Container escapes or shared-kernel CVEs affecting the host.
Emulated devices thrashing exits under load.
Trade-offs to weigh
Isolation versus density and cold start.
Virtio speed versus emulation compatibility.
Passthrough performance versus migration flexibility.
Common misunderstandings
Calling containers “lightweight VMs” without noting shared kernels.
Ignoring cgroup/VM limits when capacity planning.
Assuming nested virtualization is free.
Wrap-up
Virtualization is multiplexing with a chosen isolation cut. Know hypervisors, exits, virtio, containers, and overcommit—and design to the virtual limit you actually bought.
Exercises
- Trust boundary. Sketch a SaaS that runs customer plugins. Where do you put VMs vs containers, and why?
- Exit budget. A guest does 100k emulated NIC operations/sec. Explain qualitatively what virtio changes in the exit path.
- Mystery p99. CPU idle, latency up. Which virtualization signals (steal, balloon, host swap) do you check first?
Topic-specific references
Key takeaway
Pick the isolation boundary for your threat model, then optimize exits and device models for performance. Containers densify; VMs contain; overcommit quietly taxes tails.