System Design Patterns
Microservices vs monolith
A seller ships inventory code; a payments team owns the card path; a mobile client still shows one Checkout button. That tap must reserve stock, price the cart, charge a card, and schedule shipping, while other teams ship features every day.
A can do all of that in one process. split the work so teams deploy and fail separately. The hard question is not "how many services?". It is whether the boundary is ready for the ops cost.
Checkout journey
Auth, routing, coarse rate limits.
Orders service
Owns the cart and order record.
Inventory + payments
Separate owners, separate data.
Compensate if a later step fails.
What actually happens
In a young product, checkout lives in one codebase: shared transactions, one pipeline, easy debugging. That is still the right call for many teams. Large shops run hundreds of services because org and reliability boundaries grew, not because day-one startups need them.
When payments must release weekly while catalog changes daily, a single release train becomes the bottleneck. Teams carve a , give it a versioned API and , then route traffic through an . Cross-service writes become a instead of a hidden join across schemas.
The failure mode is a : many repos, still one blast radius. Prefer a until friction, ownership, and on-call readiness justify extraction: often via a path, not a big bang.
Try it: choose a service boundary
Each scenario describes real forces: release cadence, data ownership, and consistency. Pick how you would bound the work, then inspect fit and the sync-hop tax before you commit to extraction.
- Fit score
- —
- Required sync hops
- —
- Path availability (est.)
- —
- Strongest fit here
- Extract service
Scorecard for this scenario
- Modular monolith · 2/5 — Still one release train; payments cadence stays blocked.
- Extract service · 5/5 — Owned API, database, SLO, and on-call for payments.
- Shared library · 1/5 — Shared payment rules in a jar still deploy with checkout.
- Event-driven split · 3/5 — Events help after boundaries exist; they do not replace ownership.
Read the scenario, pick a boundary strategy, then compare fit score and path availability.
Path availability sketch. With three nines per required synchronous hop, two hops leave about 99.80% path availability before retries. Push optional work async when the user-facing budget cannot absorb every hop.
Extraction readiness. A service boundary earns its cost when the domain has stable data ownership, a versioned contract, dashboards, and rollback—not when the codebase merely feels large.
How to choose monolith vs services
1. Who owns the model weekly? If one team still reshapes the domain together, stay modular in one deploy. Split when a domain has stable data ownership and its own release cadence.
2. Can you afford the network? Five sync hops at 99.9% each leave about 99.5% path availability before retries. Push optional work async when a 300ms page budget cannot survive every dependency.
3. Is extraction ready? Require owned data, contract tests, an , dashboard, runbook, rollback, and an on-call owner before critical traffic moves.
Simple rule of thumb
Optimize for team autonomy and blast-radius isolation, not microservice count. Start coherent; extract where boundaries are clear and ops can manage contracts, tracing, and failure domains.
In simple terms: do not split because it sounds modern. Split when one release train is slowing real product work.
Intent, forces, and consequences
Intent
Give independently owned domains their own deploy, scale, and failure boundaries when one release train is slowing real product work.
Forces
Team autonomy, data ownership, latency budgets, compliance boundaries, and operational maturity all pull in different directions.
Consequences
Faster local changes come with distributed tracing, API contracts, eventual consistency, retry storms, and more deployment surfaces.
Modular monolith
Web / API
Orders / Payments / Catalog modules
One transactional database
Extract only when friction is real ↓
Microservices
API gateway
Orders / Payments / Catalog services
Owned databases per service
Event bus
async integration
Boundary decision guide
Modular monolith
Best when one team still changes the domain model weekly.
Service extraction
Best when a domain has stable data ownership, separate releases, and on-call cover.
Shared library
Best for pure utilities; bad for domain rules that need independent release.
Event-driven split
Best when consumers can tolerate delay and need replay.
Full boundary
Best when API, database, SLO, dashboard, and rollback are ready.
Extraction readiness
Require owned data, versioned API contract, , dashboard, runbook, page-worthy alert, rollback switch, contract tests, idempotency, and a cost owner before moving critical traffic.
Distributed monolith smells
Shared database writes, synchronous fan-out for simple reads, one release train for all services, cyclic dependencies, and no independent on-call owner mean the split has not bought real autonomy.
Module inside monolith
Contract and owned data
Gateway route
Shadow reads
Canary traffic
On-call ownership
Delete legacy path
Safety valves
Parity or canary failure?
Shadow parity failure
Stay on monolith module
Canary rollback
Switch traffic back to gateway route
Monolith strengths and growing pains
Strengths
- In-process calls and straightforward transactions
- Single operational surface area
- Unified refactoring across modules
Growing pains
- Coupled release cadence
- Scaling tied to hottest slice
- Large-team merge contention without modular discipline
Decomposition by capability
By capability
Orders, payments, search: each owns APIs and data contracts. Conway's law applies: team topology influences architecture.
Bounded contexts
An anti-corruption layer translates another service’s vocabulary into the receiving service’s model. For example, a legacy system may express availability as several numeric codes, while checkout needs available, reserved, or unavailable. Translate once at the boundary and test the mapping. Spreading legacy codes through checkout makes the new service depend on internal meanings it does not own and cannot change safely.
Catalog
Descriptions, media, merchandising rules
Ordering
Cart, checkout, pricing snapshots
Shipping
Labels, carriers, tracking events
Integration typically combines synchronous APIs and asynchronous events, not one universal transport.
Inter-service communication
Synchronous
REST for broad compatibility; binary RPC where latency and contracts matter. Apply timeouts, budgets, and idempotency keys on mutating paths.
Asynchronous
Events and queues decouple producers from consumers, absorb bursts, and enable replay: pair with schema governance.
Service mesh (when complexity earns it)
A helps when policy must be uniform across many languages; smaller fleets sometimes ship the same concerns in shared libraries or gateway tiers.
Gateway and BFF
Edge handle authn/z, coarse rate limits, and routing; backends stay focused on domain logic. layers tailor aggregates per client (web vs mobile) without polluting core services.
Data ownership and sagas
Orchestrated happy path (each step owns its data)
Reserve fails
Abort the before payment: no charge, order marked cancelled or never finalized: compensation is minimal because money never moved.
Charge fails after reserve
Run compensations: release inventory reservation, cancel or roll back the order record, notify the customer: order matters to avoid stranded holds.
Choreography variants emit domain events instead of a central coordinator; same idea: explicit failure paths and idempotent handlers.
Scale and dependency math
Fan-out availability
If all five dependencies must succeed, each succeeds 99.9% of the time, and failures are independent, the illustrative path success probability is 0.999 to the fifth power, or about 99.5%. Independence is a strong assumption: shared infrastructure can cause correlated failures, while optional branches and tested fallbacks change the calculation. Use this model to identify required dependencies, then measure real request outcomes.
Latency budget
A 300ms p95 page budget with four service hops leaves little room after network, serialization, and retries. Push optional work async or precompute reads.
Ops surface
80 services × 3 environments × 4 dashboards/runbooks is 960 maintained operational artifacts. Ownership must scale with service count.
User request
Gateway
Orders
Inventory
sync hop
Pricing
sync hop
Payment
sync hop
Shipping estimate
fallback cache OK
Optional async inventory work
Event stream
Tracing across hops
Context propagation
Carry one correlation identifier across every hop: gateways, sync RPC, workers, and async callbacks. So a single user request maps to one trace.
Libraries usually inject a standard traceparent (and friends); your job is to forward headers or carrier bags and not break context at thread or queue boundaries.
Spans, logs, and tail latency
The span tree shows which dependency dominates p95/p99: often one surprise downstream or retry storm.
Emit the same trace id on structured logs so you can jump from metrics → trace → log lines without guessing timestamps.
Resilience patterns
Circuit breaker
while downstream is unhealthy; probe recovery carefully.
Retry + backoff
Retry idempotent reads and safe writes with jitter to avoid herds.
Bulkhead
A bulkhead limits the resources one dependency can consume, just as separate ship compartments contain flooding. Give a slow recommendation service a bounded connection pool or concurrency limit so it cannot occupy the slots checkout needs for payment. Isolation must include queues and retry work as well as threads. When its share fills, degrade that optional feature or reject promptly rather than borrowing unlimited shared capacity.
Verification
Contract tests
Consumers publish expectations; providers verify in CI: cheap regression signal versus brittle full-environment suites alone.
Chaos drills
Inject faults to prove timeouts and failover paths behave: start scoped (pod kill) before regional drills.
Releases
Blue / green
Blue/green deployment keeps old and new application versions available so traffic can switch between them. This gives fast code rollback only while their data and message contracts remain compatible. If the new version writes a schema the old version cannot read, switching traffic back may fail. Use additive schema changes, migration checkpoints, and a clear point after which recovery requires moving forward instead.
Canary
of traffic to the new version, watch RED metrics and errors, then widen or roll back automatically.
Feature flags
Ship binaries early but gate risky paths behind flags: deploy and exposure move on different timelines.
Evolution paths
Conway's law
Architecture mirrors communication lines: autonomous teams pair naturally with service boundaries.
Strangler migration
Incrementally route slices from legacy to new implementations behind a façade: see the dedicated topic for routing tactics.
Illustrative migrations
Service interfaces first
Mandating APIs between domains often precedes physical splits: reduces surprise when databases diverge later.
Modular monolith stepping stone
Enforce module boundaries inside one deploy before paying network costs: extract hottest seams first.
How to talk about this in an interview
Selection signal
At millions of users, choose microservices for independent domain ownership, separate scaling curves, and bounded blast radius. Choose a modular monolith when the team is small or boundaries are still changing weekly.
Failure modes
Expect network partitions, cascading retries, schema drift, and partial workflow completion. Mention timeouts, idempotency, circuit breakers, and sagas.
Observability
Require trace propagation, service-level RED metrics, ownership dashboards, and contract-test failure visibility before splitting critical paths.
Rollout strategy
Extract one bounded context, route via gateway, shadow traffic, canary users, then migrate ownership and deletion milestones with dashboards proving error, latency, and cost targets.
Production deep dive: extraction economics
Bounded-context example
Catalog ownership can become a service when product data changes on a different cadence from checkout. Keep the modular monolith until that boundary and team ownership are clear.
Team and ops cost
Every service adds on-call, CI, schema ownership, dashboards, and failure modes. Count those costs before celebrating independent deploys.
Data migration during extraction
Prefer an outbox/CDC feed from one authoritative writer; two independent application writes create an unresolvable partial-failure window. Backfill from a checkpoint, catch up, shadow reads, cut ownership, and keep a reconciliation path.
Modular-monolith evolution
Start with package boundaries, explicit module APIs, and separate schemas inside one deployable. Extract only after the module already behaves like a service.
| Evolution stage | Evidence to advance | Rollback boundary |
|---|---|---|
| Module inside monolith | Explicit API, private schema access, owner, contract tests | One deploy and transaction model |
| Remote shadow service | Response parity, independent capacity, trace coverage | Route all traffic to in-process module |
| Independent owner | Data cutover, SLO/on-call, reconciliation, old access revoked | Compatibility events preserve accepted new writes |
What to remember
Optimize for team autonomy and blast-radius isolation, not microservice count. Extract services where boundaries are clear and ops can manage observability, contracts, and failure domains. Until then, a well-modularized monolith plus clear APIs often ships faster with fewer outages than premature distribution.
Primary sources
- microservices.io: Microservices pattern: service decomposition patterns.
- Google SRE: Managing Critical State: operational cost of distributed ownership.
- CNCF Landscape guide (microservices): ecosystem building blocks.
Practice and further reading
Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.
Focus for this lesson: Evolve from a modular monolith only where bounded contexts, team ownership, scaling, or release independence repay distributed-systems cost.
Explain the decision
What evidence justifies extracting a service?
Read the reasoning
A stable business boundary plus measured coupling, deployment contention, scaling isolation, or ownership pain—not source-file size or fashion.
Apply it to a production scenario
Orders and inventory are extracted but still join each other’s tables. What went wrong?
Read one approach
The deployment split preceded data and contract ownership; introduce explicit APIs/events and migrate callers before revoking cross-boundary access.
Work through the numbers
A request synchronously crosses five services, each 99.95% available independently. Estimate path availability.
Read the worked calculation
0.9995^5 ≈ 99.75% before shared failures; fan-out can consume the end-to-end availability budget quickly.
Reason through a failure
What must service extraction preserve?
Read the recovery reasoning
Business invariants and rollback remain valid while old and new paths coexist; ownership cannot be ambiguous during data migration.
Sources and further study
Continue with Service-Oriented Architecture.