Concurrency · LLD
Producer-consumer
Producer–consumer separates “accept work” from “do the slow work.” Example: a car parks. The lot should not wait while an SMS goes out over a slow network. Gate A puts a message on a queue: “notify this number for spot 101.” A worker thread takes messages from the queue one by one.
If the queue is full, Gate A waits. That wait is . It is safer than letting an unbounded queue grow until the process runs out of memory.
The real question
How do I separate “create work” from “do work” without losing the work or the lot?
The queue
Two jobs, one bounded queue
Gate / lot NotificationWorker
| |
| bounded queue |
| put() -----> take() |
| [ | | ] |
| full → wait |The queue is the shared state. You do not share the SMS client with the gate. The creates work. The does it. This is LLD, not a broker diagram.
Bound the queue on purpose
A is the design. LinkedList that only grows will take the JVM down on a slow SMS provider. When the queue is full, put() blocks or offer() returns false. Pick a size you can defend: a few hundred notifications, not “as many as memory.”
put() waits
Honest back-pressure. park() may stall if the worker is dead. Say that trade-off: occupy is correct, notify is coupled again at the limit.
offer() returns false
Occupy succeeds. Notify is dropped or logged. The driver has a ticket; the SMS may never leave. Retry from a dead-letter later. Pick one and say it.
A queue with a capacity limit
Java’s BlockingQueue is this pattern as a type. ArrayBlockingQueue is bounded and simple. The producer calls put. The consumer calls take. Do not invent wait/notify unless the interviewer asks you to. Do not let park() also send the SMS.
class NotificationBus {
private final BlockingQueue<Sms> queue = new ArrayBlockingQueue<>(256);
void enqueue(Sms sms) throws InterruptedException {
queue.put(sms); // waits if full
}
boolean tryEnqueue(Sms sms) {
return queue.offer(sms); // false: queue full, occupy already done
}
void runWorker() throws InterruptedException {
while (true) {
Sms sms = queue.take();
sms.send(); // slow I/O stays here
}
}
}What belongs on the note
Put an immutable snapshot: ticket id, phone, spot number, Instant. Do not put a live Ticket the gate might still mutate. That is confinement’s hand-off rule. The worker owns the send. The gate owns occupy. The queue owns the buffer.
Logging, task scheduling, and in-process pub-sub are the same shape. A logger produces lines; a writer consumes. A scheduler produces due jobs; workers consume. Name the bound and the full policy each time.
Stopping the worker
A is a special sentinel message that means “stop.” Enqueue it on shutdown, let the worker finish the current SMS, then exit. Thread.interrupt() during take() also works if you handle InterruptedException and restore the flag. Do not kill the process with leftover queue messages unless you accept lost receipts.
private static final Sms STOP = new Sms(null, null);
void shutdown() throws InterruptedException {
queue.put(STOP);
}
void runWorker() throws InterruptedException {
while (true) {
Sms sms = queue.take();
if (sms == STOP) return;
sms.send();
}
}Several consumers, still one queue
Two SMS workers taking from the same bounded queue is a tiny pool. Each take() gets a different note. Do not also share a mutable SmsClient without its own story — many HTTP clients are safe, many socket wrappers are not. If order of receipts must match park order, one consumer. If order does not matter, more consumers raise throughput until the provider rate-limits you. The bound still protects memory.
A logging framework is the same pattern: many producers, one (or few) file writers, a capacity limit, and a drop or block policy when the disk is slow. Task schedulers enqueue due work; workers consume. Name the capacity in every case. Unbounded “for now” is how a Saturday stall becomes an out-of-memory on Monday’s postmortem.
offer, put, and the gate’s face
If park() must return a ticket even when notify is overloaded, offer() and a metric for dropped SMS. If a missed receipt is worse than a slower gate, put() and accept the stall. CallerRuns on a pool is put() in disguise: the gate does the work. There is no secret third option that is both “never stall occupy” and “never drop a receipt” without another buffer (disk, another process). In LLD, pick one and defend it.
Ticket first
offer(), drop or dead-letter, occupy stays fast. Driver has a slip. SMS may lag.
Receipt first
put() blocks. Gate queue grows at the barrier. Memory stays bounded. Saturday feels slow.
Common mistakes
An unbounded pile of notes
LinkedList that only grows will take the JVM down on a slow SMS provider. Bound the queue so a stuck consumer stops the producer.
The producer doing the work
park() that also sends email couples the gate to the network. Drop work on the queue. Let a worker fail and retry without blocking occupy.
Hand-rolling wait/notify
BlockingQueue already is the monitor plus the array. Inventing wait/notify in an LLD interview burns time and usually misses the bound.
This is not Kafka
If the interviewer asks “what if the process dies with messages still in the queue?” you have left LLD. An in-process queue is RAM. A crash loses those messages. Durable notify needs a store or a broker — that is HLD. Until that pivot, do not design partitions. Design a capacity, a worker, an immutable message, and a full-queue policy. Saying “we’ll use Kafka” in a parking-lot machine-coding interview skips the seam they wanted to see: park() does not send SMS.
In-process pub-sub is several consumers on one event. A queue is still the shared buffer. CopyOnWrite listeners are a different collection story. Pick the shape: work to do (queue) versus many observers of a fact (listeners). Mixing them — a queue of live Tickets plus listeners mutating them — is two races.
Exercises
- Capacity 8, worker stuck, nine parks. Contrast put() versus offer() for the ninth receipt. What does the driver experience?
- The note holds a live Ticket. After put, the gate calls ticket.markClosed(). What can the worker print?
- Two workers, one STOP pill. What happens to the second worker? How do you shut both down?
- Logs and SMS share one unbounded LinkedList “bus.” Which failure domain did you merge, and what bound would you split into?
On the board, write put versus offer, the capacity, and who feels a full queue: the driver (if park blocks) or the receipt (if notify drops). That trio is the design. A queue without those three decisions is a list with extra ceremony.
If they ask about durability, say the queue is RAM and a crash drops messages. Then stop. Do not draw partitions unless they pivot the interview toward distributed systems.
Read next
Where the queue design still fails
A dead worker with a full queue blocks every park() if you used put(). A worker that swallows exceptions silently drops receipts forever. A note that holds a live Ticket races after hand-off. Two buses, two queues, no metric — you cannot tell which queue filled. Interrupt during sms.send() without a defined retry leaves a half-sent provider call; prefer finishing the send or recording a dead-letter id, then exiting.
Duplicate SMS on retry is usually acceptable; duplicate occupy is not. Keep those failure domains separate. The queue protects memory. It does not protect occupancy. Occupancy was already decided on the gate’s thread.
- Instrument queue depth and dropped offers. A silent bound is a hidden outage.
- One poison pill per consumer, or a shared running flag plus interrupt.
- Do not share the queue with a second unrelated job type without naming both producers.
Takeaway
Separate create-work from do-work with a bounded queue. The queue is the shared state. Name the full-queue policy. Make the message immutable. Keep occupy off the network. This is an in-process queue, not a distributed log.
put() makes the driver wait. offer() may drop the receipt. There is no third magic that is both never-stall and never-drop without another buffer. Pick one, instrument depth, shut down with a pill per worker.
Logs, schedulers, and notifiers use the same pattern. Unbounded “for now” is how a slow disk becomes an out-of-memory. Bound the queue while you still remember why it exists. Capacity, full-queue policy, immutable message, one worker or a named handful.
Duplicate SMS on retry is acceptable. Duplicate occupy is not. Keep those domains on different threads and different types.