System Design · Security

Zero Trust

An engineer joins a morning stand-up from a café, opens the admin console, and expects the payroll app to open, without granting their laptop a free tour of every VLAN behind the old corporate .

moves enforcement to identity-aware proxies, mesh gateways, data policies, and continuous telemetry. Phishing will succeed somewhere; lateral movement should still fail.

Per-request journey

Prove the human session.

Patch, encryption, EDR signals.

Allow, deny, or step-up.

Workloads prove identity too.

What actually happens

Instead of “VPN then trust the LAN,” an collects user identity, , app sensitivity, and risk. A returns allow, deny, or step-up . Every decision should feed SIEM.

Services talk over with workload identities. and shrink blast radius when one laptop or token is compromised. replaces broad network tails with per-app tunnels.

Rollout is platform work over months: discover flows, observe-only policies, enforce by risk tier, keep short-lived and audited. Measure denied-by-default coverage and mean time to revoke.

Try it: run a Zero Trust policy decision

Combine SSO session freshness, device posture, and app sensitivity. The policy decision point returns allow, deny, or step-up before traffic reaches the workload.

MFA completed recentlyDevice patch and encryption OKApp sensitivity

Decision: Not evaluated
MFA fresh: Yes
Device posture: Compliant
Sensitivity: standard

Signal collection

Context

SSO session + MFA freshness
Device compliance telemetry
Requested app: Standard

Policy decision point

PDP

Waiting for evaluation

Allow, deny, or step-up with reason code for SIEM.

Enforcement point

Proxy / mesh

No enforcement until policy runs.

Set identity and device signals, then run policy. Zero Trust evaluates every access attempt; network location alone does not grant trust.

Signals. User, device, app sensitivity, and risk feed the PDP on each request, not only at VPN connect time.

Enforcement. Proxies and mesh gateways apply the decision; services still mutual TLS to each other inside the mesh.

How to choose Zero Trust controls

1. Who and what? Inventory users, services, apps, and data classes: gaps become the roadmap.

2. Where is enforcement? Proxy for humans, mesh for services, row policy for data: name the owner of each decision.

3. How do you recover? Break-glass, revocation drills, and control-plane outage behavior matter as much as happy-path SSO.

Simple rule of thumb

Verify continuously, encrypt everywhere, prefer per-app access over flat VPN trust, and assume breach already happened somewhere in the fleet.

In simple terms: stop giving the whole building to whoever unlocked the front door.

Core principles

Castle-and-moat vs continuous verification

Legacy perimeter

A VPN authenticates entry to a network tunnel; what the user can reach afterwards depends on segmentation and application policy. In a broadly connected network, one compromised device may then contact many unrelated services. Zero-trust architecture removes network location as sufficient grounds for access and evaluates the requested resource explicitly. A VPN can still be one transport component when its presence does not bypass those decisions.

Zero Trust mental model

Authenticate + authorize each session; encrypt everywhere; collect telemetry for anomaly response; assume breach already happened.

Verify identity

Strong SSO, MFA tiers, workload identities for services, not shared bootstrap secrets.

Verify devices

Patch compliance, disk encryption, MDM enrollment, EDR heartbeats feed trust scores.

Verify context

Geo velocity, time-of-day risk, peer anomalies: drive step-up challenges dynamically.

Least privilege

Scoped roles, just-in-time elevation, short-lived tokens per task, not eternal admin VPN.

Encrypt everything

TLS/mTLS internally + externally: plaintext east-west traffic is technical debt.

Per-request verification loop

Zero Trust is a control plane plus enforcement points

Diagram text: the exact Mermaid definition below preserves the nodes, labels, and connections. Enable JavaScript to view the rendered diagram.

Read the diagram definition

sequenceDiagram
    autonumber
    participant User
    participant Device
    participant Proxy as Identity-aware proxy
    participant PDP as Policy engine
    participant App
    participant SIEM

    User->>Proxy: Request application session
    Device-->>Proxy: Device cert, posture, EDR status
    Proxy->>PDP: User, device, app, action, risk context
    PDP-->>Proxy: Allow, deny, or step-up decision
    alt allowed
        Proxy->>App: Forward request with signed identity context
        App-->>Proxy: Response
    else risky
        Proxy-->>User: MFA, remediation, or deny
    end
    Proxy->>SIEM: Log decision and signals

Signal collection

Policy can use verified user or workload identity, device management status, the requested action, and resource sensitivity. Each signal has a source and a freshness limit: a device declared compliant yesterday may be compromised today. Treat missing or stale evidence deliberately, especially for privileged operations. More signals do not automatically mean a better decision if their provenance is weak or their failure behavior is undefined.

Policy decision

The policy decision point evaluates the access request against current rules, while a policy enforcement point actually permits or blocks traffic. Keep that distinction visible: a central service returning deny is ineffective if a direct backend route bypasses the proxy. A step-up result can require stronger proof before a sensitive action, and the enforcement point must verify that proof before continuing.

Audit loop

Every decision feeds SIEM and risk scoring so access tightens after anomalies.

Policy decision capacity math

150k employees × 40 app sessions/day × 20 policy checks/session = 120M decisions/day, or ~1.4k average decisions/sec. At 10× peak, budget ~14k decisions/sec. If each decision log is 2 KB, daily raw audit volume is ~240 GB before replicas, indexes, and retention tiers.

Control placement threat model

Where verification sits for humans, services, devices, and data access

Place controls on choke points

Human apps: identity-aware proxy checks user, device posture, app sensitivity, and risk before forwarding.
Service calls: mesh or gateway enforces workload identity, method, path, and namespace boundaries.
Data stores: database roles and row policies defend against bypasses around application middleware.

Scale and telemetry

Large systems cannot send every request through an unlimited central policy bottleneck. Cache carefully scoped decisions when their freshness contract permits it, with a key that includes identity, resource, action, and relevant policy version. A five-minute allow cache can delay a revocation by five minutes unless invalidated. Retain complete required records for sensitive actions and measure policy lookup latency, stale decisions, and propagation time.

Latency budget: Keep policy checks near the caller; fall back to deny or safe read-only behavior for critical apps.
Audit budget: Decision logs need actor, device, resource, action, policy version, and reason code for replayable investigations.

Zero Trust threat model grid

Actor	Asset	Entry point	Lateral path	Enforcement	Detection	Residual risk
Compromised laptop	Internal apps	Phished browser session	Flat app network	Access proxy + device posture	New device, risky location, step-up failures	Approved device sessions can still be hijacked before risk signals update.
Stolen service token	Service API	Leaked workload secret	Shared namespace routes	Gateway or mesh authz	Audience mismatch, abnormal method mix	Short-lived tokens reduce but do not eliminate replay windows.
Malicious insider	Admin console	Valid SSO session	Overbroad role grants	JIT admin + recorded session	Policy change and bulk export alerts	Legitimate approvals can still permit harmful actions without four-eyes review.
Vulnerable service	Database rows	Remote code execution	Service account reuse	Database roles + row policy	Unusual query volume and deny spikes	Emergency exceptions and legacy protocols can reopen lateral paths.

Make it concrete

Say which enforcement point owns each decision. The proxy gates user sessions, the mesh gates service calls, the app enforces object rules, and the data tier adds final least-privilege checks. Then describe how all decisions are logged and revoked.

per-app proxy access

Trust scores replace VPN tunnels

VPN contrast

Traditional VPN grants broad network reach once authenticated: stolen session equals scanner-friendly interior.

Access proxy pattern

An identity-aware proxy authenticates the caller and evaluates access to one application instead of granting broad subnet reach. The application should accept traffic only through the intended trusted path or independently enforce equivalent checks. Bind forwarded identity to an authenticated connection and remove spoofable inbound identity headers. The application still owns record-level rules that the proxy lacks enough business context to evaluate.

Trust score ingredients (illustrative)

Managed device certificate present

OS patch level within SLA

Disk encryption + screen lock

MFA completion + low fraud signals

Scores gate sensitive apps: low posture routes users through remediation portals instead of silent failures.

mTLS for east-west traffic

Both peers authenticate cryptographically

Plain TLS server-auth

Client trusts server cert; server often trusts network location: insufficient for Zero Trust interiors.

Mutual TLS

Each workload presents an X.509/JWT workload identity document; service-mesh proxies enforce identities without app code duplicating TLS stacks.

Mesh authorization intent (mesh-shaped)

Select workloads via labels (e.g., app=payment-service).
Allow only cluster.local/ns/orders/sa/order-svc principals to POST /api/charge.
Everything else defaults deny: policy-as-code stored in Git with peer review.

Micro-segmentation & privilege boundaries

Networks + IAM both shrink blast radius

Segmentation

orchestrator NetworkPolicies, cloud security groups, and mesh auth combine L4 segmentation with L7 intent: payment tiers never accept arbitrary RPCs from marketing namespaces. Network segmentation narrows which workloads can connect, while application-layer policy narrows which authenticated operations they may perform. For example, a reporting service may reach a read API but should not inherit payment mutation rights simply because both run in one cluster. Roll out observed flow rules carefully so necessary background jobs and recovery tools are included without allowing unrestricted lateral access.

Least privilege IAM

Scoped cloud roles per serverless function, short-lived tokens for CI/CD, break-glass workflows audited: avoid god-role service accounts shared across teams.

ZTNA vs legacy VPN

Per-application brokerage vs network tails

Classic VPN drawbacks

Broad network visibility post-login.
Hairpin traffic patterns hurt latency.
Hard to inspect encrypted tunnels for insider threats.

ZTNA advantages

Direct-to-app tunnels with policy per resource.
No carte-blanche port scans inside corp nets.
Easier integration with IdP risk signals.

Re-verify after risky signals: geo hops, impossible travel, midnight database dumps, admin role grants.

Rollout roadmap

Phased investment beats big-bang outages

Phase 1: Discovery

Map services/data flows, classify sensitivity, instrument baseline telemetry: often months of inventory hygiene.

Phase 2: Identity spine

SSO + MFA everywhere; bootstrap workload identity and mTLS for critical namespaces; centralize policy definitions (policy languages).

Phase 3: Segmentation

Default-deny network + mesh policies; kill lateral paths verified redundant; automate policy tests in CI.

Phase 4: Detection loop

Behavioral analytics, purple teaming, automated remediation hooks: close gaps surfaced by Zero Trust telemetry.

Rollout safety plan

Discover flows first: users, service calls, data stores, admin paths, and legacy protocols.
Run observe-only policies and compare would-deny events against real business traffic.
Stage enforce mode by low-risk apps, then sensitive apps, then admin and data access.
Keep break-glass access short-lived, reviewed, and recorded with automatic expiry.
Expire legacy exceptions and test policy changes in Continuous Integration (CI) before deploy.

Case studies

Supply-chain compromises, VPN blast radius, and what per-app proxy patterns prove in production

Supply chain · nation-state class · 2020

trusted update compromise: signed binary updates as lateral movement vehicle

Attackers compromised build pipelines for monitoring software: customers auto-installed trojanized binaries signed with legitimate release keys. Detection lagged because telemetry pipelines trusted the update channel implicitly.

Beginners internalize Zero Trust as continuous verification: VPN placement means nothing if malware arrives labeled “mandatory upgrade Tuesday.”

Simplified attack / failure chain

Stealthy commits alter build scripts producing benign-looking binaries.
Signed updates propagate via trusted distribution channels.
Beacon blends into normal monitoring traffic: living-off-the-land dual-use tooling.
Endpoint telemetry discoveries pivot the full incident response effort.

Advanced controls & design pressure

provenance-inspired provenance attestations comparing reproducible artifacts.
Egress allow lists on build agents + locked-down package proxies.
workload identity identities per workload limiting unnamed lateral RPC.
Split admin domains with hardware-backed MFA + recorded sessions.
Behavioral analytics on unusual child processes, not only Indicator of Compromise (IOC) hashes.

Aftermath, signals & interview questions

Disclosure rules tightened; boards now ask quarterly dependency inventory maturity, not checkbox questionnaires alone.

Critical infrastructure · ransomware · 2021

critical infrastructure operator: legacy VPN creds + flat OT adjacency

Operators halted fuel flows after ransomware disrupted billing and SCADA-adjacent IT: physical infrastructure survived but business logic demanded cautious shutdown. VPN-accessible heritage systems amplified blast radius.

Beginners map VPN trust to implicit LAN privileges: attackers pivot faster when DHCP scopes resemble flat campus networks from the 1990s.

Simplified attack / failure chain

Credential reuse or leaked VPN account enters corp IT.
Ransomware spreads via SMB/RDP classics absent micro-segmentation.
Operators proactively isolate OT interfaces: business halts pending certainty.
Federal response + public panic illustrate cascading dependencies beyond pipes.

Advanced controls & design pressure

ZTNA + device posture for remote ops, not carte-blanche internal routing.
OT segmentation gateways with unidirectional diodes where feasible.
Immutable backups + offline recovery drills measured in hours-to-barrel-flow.
Break-glass VPN profiles requiring VP approval + recorded shells.
Public-private intel sharing channels for ICS-specific IOC feeds.

Aftermath, signals & interview questions

sector directives reshaped pipeline cybersecurity reporting timelines.
Interview: “Which OT-adjacent VLAN still trusts DHCP laptops?”: prepare honest diagrams.

Research · enterprise access model

per-app proxy model: publishing internal apps without trusting the corporate network

A large enterprise operationalized internal apps reachable from anywhere: policy enforced via device inventory, user risk signals, and proxied TLS, not assumed LAN confidentiality.

Continuous evaluation means access is reassessed at appropriate enforcement points and when meaningful risk or policy signals change. It need not mean an expensive synchronous query to every identity and device database on every HTTP request. Define cached-decision lifetimes, session termination signals, and step-up triggers so the actual revocation delay matches the risk of the application.

Simplified attack / failure chain

Legacy attackers relied on campus DHCP giving trusted routes.
per-app proxy model inserts access proxies issuing per-session authorization tickets.
Device posture stale or jailbroken ⇒ downgrade or deny gracefully.
Analysts replay logs using correlated user/device/app tuples, not NAT IPs alone.

Advanced controls & design pressure

Unify MDM + cert lifecycle automation before policy rollout.
Strong identity-aware proxies fronting legacy JVM apps lacking native OIDC.
Canary cohort migrations measuring productivity friction metrics weekly.
Integrate threat intel into policy: geo velocity, impossible travel, malware signals.
Pair with runtime-enforcement-style runtime enforcement: uniform TLS + ACL everywhere.

Aftermath, signals & interview questions

Every security pitch may use “Zero Trust”: know the primitive controls, not marketing gloss.

Identity provider · extortion crews · recurring theme

Identity-provider support-centric breaches: helpdesk as weakest MFA gate

Extortion gangs repeatedly targeted identity support teams and downstream tenants via social engineering: convincing support reps to reset MFA or add rogue factors using polished caller scripts and leaked employee directories.

Beginners acknowledge Zero Trust fails without people-tier controls: support consoles wield god-mode if approvals lack out-of-band verification.

Simplified attack / failure chain

OSINT harvests employee names + helpdesk queue etiquette.
Attacker poses as locked-out admin citing faux exec urgency.
MFA factors reset or backup codes leaked via shadow IT channels.
Attacker pivots OAuth grants across application portfolio: ransom note follows.

Advanced controls & design pressure

Hardware passkey protocol for Tier-0; kill SMS fallback pathways.
JIT admin roles with mandatory manager approval voice loops.
Immutable SIEM pipelines watching IdP config deltas per minute.
Customer-verifiable encryption / split-knowledge escrow for break-glass.
Purple-team drills specifically targeting support chat workflows quarterly.

Aftermath, signals & interview questions

Cyber-insurance questionnaires now demand recorded evidence of support-console MFA, not checkbox attestations.

Hospitality · ransomware ops · 2023

hospitality disruption archetype: social engineering + flattened identity pathways

High-profile hospitality breaches featured rapid operational paralysis: slots, reservations, and payments degraded: after attackers blended phishing/vishing against IT helpdesks with ransomware deployment across interconnected properties.

Beginners translate hospitality outages into architecture lessons: guest Wi‑Fi, point-of-sale systems, and back-office SSO often share brittle identity assumptions: Zero Trust demands segmentation between customer networks and crown-jewel payroll.

Simplified attack / failure chain

Social engineer obtains elevated IT credentials via helpdesk pathways.
Ransomware affiliates deploy encryptors + data theft tooling rapidly.
Business continuity toggles “manual mode” costing millions daily.
Forensics spans months: brand damage exceeds ransom deliberations.

Advanced controls & design pressure

Separate IdP tenants/environments for guest vs corp vs OT bridges.
Mandatory phishing-resistant MFA on any account touching payment-security settlement.
ZTNA exposing admin consoles only via hardened privilege workstations.
Immutable backups + offline runbooks for reservations/payments.
War-room comms integrating legal/PR earlier: disclosure clocks ticking.

Aftermath, signals & interview questions

Boards now budget identity segmentation alongside flashy firewall refreshes: great anecdote for system-design interviews tying UX to blast radius.

Audit deepening: continuous evaluation and break-glass

Continuously evaluate identity and device posture, issue short-lived workload identities, and define break-glass access with audited expiry. Measure rollout success with denied-by-default coverage, control-plane outage behavior, and mean time to revoke compromised credentials. Break-glass access is a narrowly scoped emergency path for recovering when ordinary identity or policy infrastructure is unavailable. Keep it independently usable, strongly authenticated, short-lived, and audited, with alerts and a review after use. Test it during drills. An untested emergency account may fail during recovery, while a permanent all-powerful bypass can undo the isolation the normal architecture provides.

What must never become false

Network location (VPN, office LAN, private subnet) must never be sufficient authorization by itself. Every human session and service call must prove identity + device/workload posture + policy for that app or RPC, and compromised credentials must be revocable within a measured window.

Failure timeline: VPN trust vs lateral movement

T+0      Phished engineer completes VPN MFA; laptop lands on flat corp VLAN
T+20m    Attacker scans RFC1918, finds unauthenticated admin UI on "internal" IP
T+45m    Service account shared across namespaces accepts spoofed private-IP calls
T+2h     Payroll DB dump starts; SIEM only sees "VPN user online"
T+1d     ZTNA + mTLS would have required per-app + workload identity: VPN alone failed
Outcome  Invariant broken: interior network membership treated as authorization

Counterexample to "we have MFA on the VPN": MFA at the tunnel does not authorize every east-west hop afterward.

Exercises

150k employees × 40 sessions/day × 20 PDP checks/session ≈ decisions/day and average decisions/sec. At 10× peak and 2 KB logs, what is daily raw audit volume?
Migrate one Tier-0 admin app from VPN to ZTNA. Write the observe-only → enforce stages and the break-glass rule that preserves the invariant during IdP outage.
Payment pods accept RPCs from any cluster IP today. Design mTLS + mesh authz so marketing namespaces cannot call /api/charge even on the same VPC.

Primary references

Zero Trust architecture primary sources.

NIST SP 800-207: Zero Trust Architecture Google: BeyondCorp research papers SPIFFE: Workload identity

Short takeaway

Perimeters evaporated with cloud apps and remote work: Zero Trust aligns spend with actual risk by verifying each hop. It is expensive and cultural, but cheaper than one ransomware tour through a flat legacy VLAN.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Continuously authorize user and workload access from identity, device, resource, context, and policy without assuming network location is trust.

Explain the decision

Does mutual TLS alone implement zero trust?

Read the reasoning

No. It authenticates endpoints; authorization, device/workload posture, least privilege, continuous evaluation, audit, and lifecycle remain necessary.

Apply it to a production scenario

The policy control plane is down. Should production traffic fail open?

Read one approach

Define risk-tiered cached decisions and expiry: critical new privilege should fail closed, while already-authorized low-risk traffic may use short last-known-good policy.

Work through the numbers

Policy evaluation adds 4 ms to a 120 ms p99 budget. What percentage is consumed?

Read the worked calculation

About 3.3%; include cache misses and control-plane tail latency rather than only average cost.

Reason through a failure

What must break-glass access guarantee?

Read the recovery reasoning

It is narrow, time-bound, strongly authenticated, independently logged, immediately reviewable, and revocable without the failed normal control plane.

Sources and further study

Continue with Access Control Models.