SwadhyAI Notes

All System Design notes

System Design Data

Concurrency Control Algorithms

Isolation levels say which anomalies you refuse. Concurrency control is the machinery that refuses them: lock tables, version chains, and commit-time validation. Two engines can both claim Repeatable Read and still queue, abort, or vacuum for completely different reasons.

This note complements Transactions & Isolation. It does not retell dirty reads and write skew. It compares , , and as schedulers: who waits, who aborts, what memory they pin, and how deadlocks and leases show up once you leave a single process.

Inventory row, three schedulers

1

Second writer waits on X. Queues form; deadlocks are possible.

2

Readers keep going on old versions. Writers still collide on the live row.

3

Both write, one fails commit validation and retries.

4

Pick from contention

Hot row → locks or a single sequencer. Read-mostly → MVCC/OCC.

What actually happens

Every concurrent database is a scheduler plus a recovery log. The scheduler's job is to produce a history equivalent to some serial order for the conflicts you care about. Pessimistic algorithms reserve the right to touch a row. bets that reservation is wasted work. changes the object of conflict: readers consume a version, writers create a newer one, but two writers of the same key still need a rule.

Inside the engine, protect pages for the duration of a physical operation. Transaction locks protect logical rows across user think-time. Mixing those words in an interview is a tell that the mechanism is fuzzy.

At scale the same three families reappear: ZooKeeper-style exclusive znodes are 2PL with leases; etcd compare-and-swap is OCC; replica snapshot reads are MVCC. The algorithms did not stay in textbooks.

How to choose a scheduler

1. How hot is the same key? High write conflict favors pessimistic locks or a single-threaded owner. OCC abort rate is the giveaway that the bet failed.

2. Are reads long? Reports and snapshots want versions so they do not pin writers. Then you must bound snapshot age or undo/vacuum explodes.

3. Can the client retry? OCC and move pain to the application. If a side effect already charged a card, optimism is not free.

Simple rule of thumb

Default OLTP: MVCC plus short transactions, row locks on the contested write. Hot counters: atomic increment or a sequencer. Read-mostly with rare conflicts: OCC. Distributed mutual exclusion: lease plus —never a TTL lock alone.

Two-phase locking, then deadlock

Strict 2PL on one row

Diagram text: the exact Mermaid definition below preserves the nodes, labels, and connections. Enable JavaScript to view the rendered diagram.
Read the diagram definition
sequenceDiagram
    autonumber
    participant A as Txn A
    participant L as Lock manager
    participant B as Txn B

    A->>L: X lock row 42 (growing)
    B->>L: X lock row 42
    L-->>B: Wait
    A->>L: COMMIT, release (shrinking)
    L-->>B: Grant X
    B->>L: COMMIT, release

Wait-for cycle

Diagram text: the exact Mermaid definition below preserves the nodes, labels, and connections. Enable JavaScript to view the rendered diagram.
Read the diagram definition
flowchart LR
    A[Txn A holds 42, waits 7]
    B[Txn B holds 7, waits 42]
    A -->|wait-for| B
    B -->|wait-for| A

Strict holds X locks until commit so aborted writes cannot become other people's reads. Conservative 2PL grabs everything at start (rare in SQL; closer to some stored-procedure designs). The lock manager stores a per resource and a wait queue. Compatibility: S with S is fine; S with X is not.

plaintext
Compatibility (same resource):

           requested S    requested X
  held S     yes            no
  held X     no             no

Intention (granularity):
  IS  I will take S on a child (row)
  IX  I will take X on a child
  SIX table is S for me, and I will X some rows

  Table X is incompatible with IS/IX/S/X of others.
  That is how ALTER TABLE waits out row workers.

Granularity, gap locks, and escalation

Fine locks raise concurrency and lock-table RAM. Coarse locks are cheap to track and brutal under mixed workloads. Predicate and gap (next-key) locks close the hole where an insert would create a phantom—the mechanism behind some Repeatable Read engines, discussed as a user-visible anomaly in the isolation note.

GranuleConcurrencyRisk
RowHigh, if keys are distinctHot row still serial; many locks per scan
PageMediumFalse conflicts on unrelated rows sharing a page
Table / partitionLow can freeze OLTP during a batch
Gap / predicateProtects rangesInserts into the gap wait; deadlock shapes surprise people

are the bookkeeping that lets row X and table S coexist safely. Skip them in a homemade lock manager and you will either scan all child locks on every table request or take table locks for everything.

Deadlocks: detect, timeout, or prevent

Cycle detection walks the wait-for graph and aborts a victim (often the cheapest undo, or the newest transaction). Lock wait timeouts are the blunt tool: they also fire on congestion, so they are a poor primary detector. Prevention schemes assign timestamps: and wound-wait never create cycles. They abort more often and need retries, which is acceptable in distributed lockers where building a global graph is the expensive part.

Application habits that actually reduce deadlocks: one lock order (always accounts before inventory), smaller transactions, and SELECT … FOR UPDATE on the exact rows you will write instead of locking a huge range “just in case.”

MVCC as a version scheduler

The isolation note explains what a snapshot means to the user. Mechanism view: each row version carries a creator id and a retire id (or an undo pointer to the previous image). A reader's snapshot is a cut in transaction-id space: versions committed before the cut are visible; in-progress creators are not. Writers typically still take a short exclusive lock or use first-updater-wins so two updaters do not install two “latest” versions.

plaintext
Visibility sketch (append-only tuples):

  snapshot = {xmin, xmax, active_xids}
  version visible if
    creator committed before snapshot
    and not retired before snapshot
    and creator not in active_xids

  Reader never waits for writer's X — it sees the old version.
  Writer vs writer: still a conflict on the live key.

Costs the isolation note only sketches:
  - old versions live until the oldest snapshot releases them
  - freeze / wraparound / undo exhaustion are scheduler debts
  - indexes may point at many versions of one key
ReadersProgress during writes; long readers pin history.
WritersCreate versions; still serialize per key.
OperatorWatch bloat, oldest xmin, undo, and long transactions.

Optimistic validation and timestamp order

Diagram text: the exact Mermaid definition below preserves the nodes, labels, and connections. Enable JavaScript to view the rendered diagram.
Read the diagram definition
flowchart TB
    R[Read phase: copy versions, no row X locks]
    V{Validate write set vs committed writers}
    W[Write phase: install versions]
    AB[Abort and retry]
    R --> V
    V -->|no conflict| W
    V -->|conflict| AB
    AB --> R

Classic (Kung–Robinson): read phase, validate phase, write phase. Validation checks that the read set is unchanged and that write sets of overlapping committed transactions do not intersect in a way that breaks serial order. HTTP If-Match / version columns are the same idea in APIs. is OCC sitting on MVCC snapshots: the dangerous pattern is often a read-write antidependency cycle, aborted at commit instead of locked up front.

assigns each transaction a time and rejects operations that arrive “too late” for that time. The Thomas write rule lets a late write be ignored if a later timestamp already overwrote the item—another optimistic flavour. These schedulers shine when conflicts are sparse. On a single popular seat they become retry storms, which is why production systems hybridize: MVCC for reads, locks or compare-and-set for hot writes.

Compare-and-set as pocket OCC

sql
UPDATE seats
SET holder = $user, version = version + 1
WHERE id = $seat AND version = $seen AND holder IS NULL;

0 rows → someone else won; reread and decide.
This is OCC without a full validator: the predicate is the validate phase.

Why engines hybridize

Pure 2PL blocks readers behind writers. Pure OCC thrashes on contended keys. Pure MVCC still needs a write-write rule and garbage collection. Real engines: versions for readers, brief X or first-writer-wins for the live tuple, optional SSI for serializable, explicit SELECT FOR UPDATE when the app already knows the hot row.

Worked example: last seat, three mechanisms

Two checkouts, one seat. Isolation note: this is lost update / write skew depending on the SQL shape. Mechanism note: pick the scheduler and predict waits versus aborts.

plaintext
2PL:
  A: X lock seat, set holder=A
  B: waits
  A: COMMIT
  B: wakes, sees taken, returns sold-out
  User-visible: latency spike on B, no retry logic required

OCC / version column:
  A and B both read version=1, empty
  A: UPDATE ... version=1 → 1 row, COMMIT
  B: UPDATE ... version=1 → 0 rows → retry or fail
  User-visible: B aborts fast; retry may lose again under load

MVCC snapshot + write conflict:
  Both read old version (available)
  First writer installs new version
  Second writer hits cannot-serialize or lock wait
    depending on isolation and engine
  User-visible: mix of snapshot reads and a write-write rule

Distributed lock without fence:
  A holds Redis key 30s, GC pause 35s
  B takes lock after TTL, writes
  A wakes and also writes  → split brain
  Fix: fencing token checked by the storage that accepts writes

Distributed locks, leases, and fencing

A TTL in a cache is not a lock. Pause, GC, and network delay all let two holders believe they are exclusive. A (or epoch, or lock generation) must be stored with every write the lock was meant to protect, and the store must reject stale generations. Chubby, ZooKeeper, and etcd-style systems pair leases with session liveness. If you cannot put the token in the write path, you do not have mutual exclusion—you have a best-effort hint.

Prefer the database's scheduler

If the invariant lives in SQL, use the engine: row locks, constraints, or version predicates. A Redis lock around a non-transactional read-modify-write duplicates 2PL badly and still loses on crash. Distributed locks belong to resources the database does not know about (a single leader's duty, a file on NFS).

Trade-offs to weigh

Pessimistic 2PL

Predictable: conflict becomes wait. Deadlocks, convoys, and lock-table memory are the bill. Great when a hot row must have exactly one winner and clients hate retries.

MVCC

Readers fly. Storage, vacuum, and long-transaction pinning are the bill. Write-write still serializes. Do not advertise “lock-free database.”

OCC / SSI

No wait on the happy path. Abort rate is the capacity limit. Side effects must be idempotent. Hot keys need a different path.

Coarse vs fine locks

Table locks are simple and deadly under mixed OLTP. Row plus intention locks scale until escalation or gaps surprise a batch job.

Choosing and explaining concurrency control

A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.

Explain the core idea

Isolation is the user contract; 2PL, MVCC, and OCC are schedulers that implement it.

Pessimistic waits, optimistic aborts, versions keep readers off the writer queue.

Distributed locks need fencing tokens; TTLs alone are not exclusion.

When it belongs

You must justify SELECT FOR UPDATE versus a version column versus Serializable retries.

A hot row, deadlock spike, or vacuum/undo blow-up appears in the design.

Someone proposes Redis SETNX as the source of correctness.

How it fails

Deadlock cycles from opposite lock orders.

OCC abort storms on contended keys.

Long snapshots pinning versions until disks fill.

Lock expiry without fencing, then split-brain writes.

Trade-offs to weigh

Wait-time (locks) versus retry-time (OCC/SSI).

Reader concurrency (MVCC) versus garbage-collection cost.

Fine-grained locks versus lock-table size and escalation.

Common misunderstandings

Retelling isolation anomalies instead of naming the scheduler.

Saying MVCC means no locks anywhere (latches and write-write still exist).

Treating a cache TTL as a mutex.

Retrying OCC without idempotency around payments.

Wrap-up

Pick the scheduler from conflict rate and retry policy, not from vendor slogans. Keep isolation-level vocabulary for the companion note; keep lock modes, versions, and validation here.

Exercises

  1. Same seat. Write the wait/abort story under strict 2PL, under version-column OCC, and under MVCC with write-write conflict. What does the client retry?
  2. Intention locks. Why must a row X lock take IX on the table first? What goes wrong if you skip it?
  3. Lease. Sketch a leader writing to object storage with a fencing token. Where is the token checked, and what happens after a GC pause longer than the TTL?

Primary sources

Key takeaway

Concurrency control is a scheduler: wait, version, or validate. Match it to contention and to whether the client can retry. Isolation levels name the contract; 2PL, MVCC, and OCC name the machine. Distributed exclusion without a fence is wishful thinking.