System Design Case Study
High-frequency trading systems
A tick arrives on the exchange feed. Within a few microseconds your stack must update the book, decide, pass , and put an order on the wire—or stay silent. Cloud-style "scale out and retry" thinking loses here: distance, jitter, and unsafe sends dominate.
Design for a hard , a single safe path through the , and recovery that never invents phantom orders after failover.
Tick → trade journey
Decode, gap-check, rebuild book.
Strategy decide
Signal on local state; no remote RPC.
Limits before anything leaves.
Sessioned FIX or binary; record fills.
What actually happens
Production HFT is rarely "one matching engine in Kubernetes." Venues publish market data; firms colocated in the same facility consume it with dedicated , keep a local book, and react with strategies that never call a remote database on the hot path.
Orders leave through an that owns the exchange session, applies , and records every ack, reject, and fill. Many stacks use NICs and clocks so tick-to-trade measurements are trustworthy.
Failover and are first-class: a wedged process must not keep sending, and a recovered standby must not double-submit against an unknown session state.
How to choose the design moves
1. Are you designing a venue or a participant? Connecting to an exchange is the common interview framing: feed in, gateway out. Designing an internal adds price-time priority and fairness requirements that change the whole story.
2. What is the latency class? Milliseconds allow ordinary TCP stacks and managed services. Microseconds force , pinned CPUs, and often .
3. Where does safety live? Risk and kill switches belong in the gateway path that owns the session, not only inside the strategy process that might hang.
Simple rule of thumb
Correctness of the book and safety of outbound orders beat average latency. A fast wrong book loses money; a fast unconstrained sender can lose the firm.
In simple terms: see the market honestly, decide locally, and never fire an order the risk gate did not allow.
Case-study upgrade
Realistic Case Study Math
Realistic problems are product stories with one hidden bottleneck. First find what must be exact, then calculate the stream, storage, and worker load.
Assumptions and formulas
- For collaboration, count edit operations, hot documents, snapshots, and presence fan-out.
- For marketplace and matching systems, count state transitions, candidate scans, and timeout windows.
- For telemetry and metrics, count messages/sec, series cardinality, compressed blocks, and alert evaluator work.
Concrete math
- Collaborative editor: 1B edit operations/day = 11.6K ops/sec average; a hot document with 1K editors can create thousands of presence messages/sec.
- Delivery marketplace: 100K orders/minute in a region = 1.67K orders/sec; scanning 50 candidates/order gives 83K candidate scores/sec.
- Device telemetry: 10M devices x 1 message/minute = 167K messages/sec; a 5x reconnect burst becomes 835K messages/sec.
Design decisions forced by numbers
- State-machine cases need idempotency keys, audit logs, and clear timeout transitions.
- Ingestion cases need partitioned streams, backpressure, replay, and late-event handling.
- Hot collaborative objects need a single sequencer or explicit conflict-merge policy.
Scaling to millions
- At 1M users: one region and simple queues are enough for most practice cases.
- At 10M users: isolate hot tenants, shard streams, and separate online serving from analytics.
- At 100M+ users: design replay, regional failover, quota protection, and partial-result serving.
Readable flow
Interview follow-ups to expect
- What must be exactly correct, and what can lag?
- How do retries avoid duplicate state changes?
- How large can the backlog become during a regional or worker outage?
Operational readiness checklist
After the math, explain how this capacity plan is measured, protected, launched, and evolved.
Observability
- Attach each capacity estimate to a dashboard metric: p99 latency, queue age, cache hit rate, shard skew, error rate, and freshness lag.
- Alert on rate-of-change and saturation, not only hard failures; backlogs and hot keys usually warn before an outage.
Security and abuse
- Budget for auth checks, rate limits, quota isolation, audit logging, privacy filters, and abuse-review side paths.
- Separate trusted internal events from user-controlled input so retries and backfills cannot bypass policy checks.
Degraded modes
- Define the fallback when a dependency is slow: stale cache, partial results, read-only mode, delayed async work, or admission control.
- Protect the invariant first; reduce freshness, personalization, or analytics quality before risking duplicate or corrupt state.
Testing and rollout
- Test average, peak, and hot-object traffic; include retry storms, cold caches, lagging workers, and regional dependency loss.
- Use canaries, shadow reads, dual-write validation, feature flags, and explicit rollback criteria for risky path changes.
Rollout, backfill, and runbooks
- Make migrations and backfills idempotent, resumable, rate-limited, and visible through progress and error metrics.
- Write the runbook before launch: owner, dashboards, paging thresholds, mitigation switches, replay steps, and repair commands.
10x evolution
- For realistic case study math, state which number forces the next move: cache memory, shard count, write amplification, queue lag, egress, or failover reserve.
- Add the next layer only when the metric forces it: sharding, regional reads, async replay, hot-key isolation, or stricter backpressure.
Scope you should nail down
Product surface
- Ingest one or more venue market-data feeds; rebuild L1/L2 books
- Strategy hooks that emit order intent (not raw exchange messages)
- Pre-trade risk: position, notional, price collar, message-rate caps
- Order gateway with or venue binary protocol; fills and cancels
- Monitoring, PTP-aligned telemetry, kill switch, hot-standby failover
Scale / latency targets
- Feed rates: tens of thousands to millions of updates/sec per venue (instrument-dependent)
- Tick-to-trade budget: often <50–100µs colocated for aggressive strategies; state the number you own
- Order rate limits: venue-imposed + firm-imposed; bursts must not trip throttles blindly
- Availability: trading window continuity; cancel-on-disconnect expectations with the venue
Capacity picture
Feed: 1M updates/sec × 64B average ≈ 64 MB/s wire; decode + book update must stay ahead of the NIC.
Hot path CPU: pin feed, strategy, and gateway threads; avoid GC pauses and cross-NUMA hops.
Telemetry: sampling every event can exceed the trading path—buffer asynchronously; never block send on logging.
Storage: tick history and drop-copy are offline; the trading process keeps only live book + risk state in memory.
Tick-to-trade path
Read the diagram definition
sequenceDiagram
autonumber
participant EX as Exchange feed
participant FH as Feed handler
participant BK as Book / state
participant ST as Strategy
participant RK as Pre-trade risk
participant GW as Order gateway
participant ME as Matching / venue
EX->>FH: UDP multicast tick (seq n)
FH->>FH: gap check + decode
FH->>BK: update L2 book
BK->>ST: notify delta
ST->>RK: propose order intent
alt risk reject
RK-->>ST: block + reason
else risk allow
RK->>GW: authorized order
GW->>ME: FIX / binary NewOrder
ME-->>GW: ack / fill / reject
GW-->>ST: execution report
endInterviewers want a single critical path with risk before the wire. Remote databases, object stores, and chatty RPCs do not belong on this diagram.
Failover and kill path
Read the diagram definition
sequenceDiagram
autonumber
participant A as Primary gateway
participant B as Hot standby
participant EX as Exchange
participant MON as Health monitor
participant KS as Kill / risk control
Note over A,EX: Primary owns session + sequence
A->>EX: heartbeats + orders
MON->>A: health probe
alt primary healthy
MON-->>B: stay standby
else primary fail / feed gap storm
MON->>KS: freeze new risk
MON->>B: promote standby
B->>EX: session recover / resync
B->>KS: resume under limits
Note over B,EX: Cancel-on-disconnect may clear open orders
endPromotion must reconcile sequence state with the venue. Prefer cancel-on-disconnect and a frozen risk window over racing two gateways that both believe they own the session.
APIs worth sketching
// Internal strategy → gateway (not the public exchange API)
submitIntent(strategy_id, instrument, side, qty, limit_price, client_order_id) → Accepted | RiskReject
cancelIntent(strategy_id, client_order_id) → CancelQueued | Unknown
replaceIntent(strategy_id, client_order_id, new_qty, new_price) → Accepted | RiskReject
// Control plane
armKillSwitch(scope: strategy | desk | firm, reason) → Frozen
setRiskLimits(strategy_id, position_cap, notional_cap, max_msg_rate) → Ok
getBookSnapshot(instrument) → L2 levels (ops / warm start only)
// Exchange-facing (gateway owned)
NewOrderSingle / OrderCancelRequest / ExecutionReport (FIX or binary)Core data model
Instrument
OrderBook (in-memory)
Order
Position / Risk
Execution
Session
High-level architecture
Market data and the book
Feed handler responsibilities
Subscribe to the venue channel, detect sequence gaps, request snapshots or retransmission when supported, and publish normalized deltas. A gap without recovery must mark the book stale so strategies cannot trade on fiction.
In-memory book
Price-level or order-by-order structures live in process memory on the same NUMA node as the NIC queue. Persist ticks asynchronously for research; never make the trading decision wait on disk.
Gap handling sketch
onPacket(seq, payload):
if seq == expected:
apply(payload); expected += 1
else if seq > expected:
markBookStale()
requestRetransmit(expected .. seq-1) // if venue supports
// do not trade until bookResynced()
else:
// duplicate / late — ignoreRisk, gateway, and protocols
- Position and notional after hypothetical fill
- Price collar vs last / mid / reference
- Per-strategy and firm message-rate budgets
- Instrument allow-lists and trading-hours gates
- Own / binary session and sequences
- Idempotent client_order_id mapping
- Serialize sends; apply venue throttle
- Publish execution reports to strategies + ledger
Busy-poll NIC queues, pinned threads, hugepages, and disabled frequency scaling reduce jitter. Only adopt when the proves the OS stack is the bottleneck.
Hardware timestamps on ingress and egress let you attribute delay to feed decode vs strategy vs risk. Sample aggressively; ship logs off-box asynchronously.
If you design a matching engine
Some interviews ask for an exchange-like . Keep it single-threaded for determinism on a shard of instruments, or use a sequenced input log so every replica replays the same order of messages.
Price-time priority, atomic book transitions, and fair FIFO at a price level are the invariants. Persist a sequenced input log for recovery; rebuild books by replay rather than mutating shared memory from multiple writers.
// Simplified per-instrument shard loop
for msg in sequencedInput:
match msg.type:
NEW: restOrMatch(book, order)
CANCEL: remove(book, order_id)
REPLACE: cancel+new under same rules
emitExecReports()
// checkpoint book snapshot periodically for faster restartFailure modes
Feed gap / stale book
Stop aggressive strategies; request snapshot; clear stale flag only after sync. Trading on a gapped book is a firm-level incident.
Strategy runaway
Message-rate and loss at the gateway freeze sends even if the strategy loop spins.
Primary gateway crash
Rely on cancel-on-disconnect where available; promote standby after sequence recovery; never dual-active sessions.
Clock skew
Without , latency dashboards lie and regulatory timestamps disagree. Alert on sync offset, not only on app latency.
Risk config push mid-burst
Apply limit updates atomically per strategy epoch so a concurrent check never sees a half-written config.
Trade-offs
Latency vs operability
Kernel bypass and FPGA acceleration win microseconds but raise change risk. Prefer measuring the budget first; many desks are limited by venue RTT and risk policy, not by the OS stack.
Throughput vs determinism
Multi-threaded matching increases throughput but invites races. Sequenced single-writer shards scale by instrument partition instead.
Rich logging vs hot path
Full tick capture is invaluable for research and disputes; it must be async. Dropping telemetry under pressure is better than delaying orders—or say so explicitly if compliance forbids drops.
Colocation vs cloud
is mandatory for competitive HFT. Cloud regions are fine for research, risk aggregation, and slower execution styles—not for microsecond tick-to-trade against a colocated peer.
Production readiness
SLOs and metrics
- Tick-to-trade p50/p99/p99.9 against the stated budget
- Feed gap rate, book stale duration, retransmit success
- Risk reject rate, kill-switch activations, order ack latency
- PTP offset, NIC drop counters, GC/pause time if applicable
Degraded modes
- Stale book → quote-only cancel / no new aggressive orders
- Venue throttle → paced cancels first, then new orders
- Standby promotion → freeze until session sync confirmed
Security and compliance
- Control-plane auth for kill and limit changes; audit every mutation
- Segregate research clusters from order-entry networks
- Retain drop-copy and order audit trails per regulatory retention
Choosing and explaining HFT system design
A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.
Explain the core idea
Separate market-data correctness, local decisioning, pre-trade risk, and a single session-owning gateway.
State a latency budget and put only budgeted work on the tick-to-trade path.
Explain failover with cancel-on-disconnect and no dual-active order sessions.
When it belongs
The problem mentions microsecond latency, colocated trading, or exchange connectivity.
The interviewer asks how you prevent a buggy strategy from blowing position or message limits.
You need to contrast HFT with ordinary low-latency web services.
How it fails
Trading on a gapped book after silent packet loss.
Two gateways both sending after a split-brain failover.
Risk checks running asynchronously after the order is already on the wire.
Trade-offs to weigh
Kernel bypass and FPGA vs simpler stacks that ops can change safely.
Deterministic single-writer matching shards vs parallel throughput.
Full tick capture fidelity vs never blocking the send path.
Common misunderstandings
Designing HFT as a generic microservices diagram in a distant cloud region.
Putting a shared database on the critical path for every tick.
Ignoring venue sequence numbers, throttles, and cancel-on-disconnect behavior.
Takeaway
HFT design is a latency budget plus a safety boundary. Keep the book honest, decide locally, enforce risk before the wire, and recover sessions without inventing duplicate orders.
Review the design
Feed path: gap-aware handlers rebuild in-memory books; stale books must not trade.
Hot path: strategy → pre-trade risk → session-owning gateway; no remote DB.
Latency: colocation, pinned CPUs, optional kernel bypass only when the budget demands it.
Safety: kill switches and limits enforced at the gateway, not only in strategy code.
Failover: single active session, cancel-on-disconnect, sequenced recovery.
Time: PTP-aligned telemetry so percentiles and forensics are real.
Compare a starting answer with a more complete design
A starting answer
Ingest market data, keep a local order book, decide with a strategy, check risk, send orders through a gateway, and log fills.
Developing the reasoning
Discuss colocation and kernel bypass trade-offs, gap/stale-book handling, gateway-enforced kill switches, PTP measurement, and dual-active session prevention under failover.
Deeper walkthrough
HFT deeper walkthrough
A trading participant colocated with a venue consumes market data, decides locally, and submits orders under firm risk policy. Speed matters only after correctness and safety.
Requirement refinements
Clarify instruments, venues, latency class (ms vs µs), whether you build a matching engine or only connect, and regulatory audit needs.
State invariants: no trade on stale books; no outbound order without risk; at most one active order session per gateway identity.
API design details
Expose strategy intents with client_order_id idempotency; keep FIX/binary sessions private to the gateway.
Control-plane APIs for kill switches and limit updates must be authenticated and audited.
Data model and storage
Keep live books, positions, and sessions in memory; persist drop-copy, fills, and ticks asynchronously.
Model order state transitions explicitly so cancels, partials, and rejects reconcile after reconnect.
Scaling strategy
Shard by instrument or venue channel; pin threads to NIC queues; scale research and history off the hot path.
Cap order message rates independently of feed rates so busy markets cannot auto-amplify sends.
Reliability, recovery, and operations
Feed gaps mark books stale; failover freezes risk until session sync; cancel-on-disconnect clears ambiguity when possible.
Monitor tick-to-trade tails, gap rate, PTP offset, risk rejects, and kill-switch events as separate signals.
Primary sources
- FIX Trading Community: session and application messaging for order entry.
- NIST / IEEE 1588 overview: precision time for distributed measurement.
- DPDK: user-space packet processing used in low-latency networking.