SwadhyAI Notes

All Low-Level Design notes

Concurrency · LLD

Executors and thread pools

A thread pool reuses a fixed set of worker threads instead of creating a new thread for every task. Example: if every park() does new Thread() to send a receipt, two hundred Saturday cars mean two hundred threads. The machine spends time creating stacks instead of marking spots.

A small pool of workers — two, four, or eight — taking jobs from a bounded queue is usually enough. When the queue is full, you need a rejection policy: wait, drop the job, or tell the caller the lot is busy.

The real question

How many workers, and what happens when they are all busy?

Size on purpose

A handful of cashiers, not a crowd

A is workers plus a . You submit. A worker picks up. Occupancy stays on the caller’s thread.

Workers

Fixed 2 for SMS. About cores for hashing. Name the number.

Tray

ArrayBlockingQueue of 64, not LinkedBlockingQueue with no cap.

Ceiling

Abort, CallerRuns, or drop. The policy is part of the design.

CPU-bound

wants about as many threads as cores. Extra workers queue for the same ALUs.

I/O-bound

can use a few more, because workers sleep. Still bound the pool. Still bound the queue.

A bounded team, a real no

Reason first: occupy does not wait for SMS; notify has a ceiling. Then ThreadPoolExecutor is just that sentence typed. park() itself stays on the caller’s thread.

java
ThreadPoolExecutor notifyPool = new ThreadPoolExecutor(
    2, 2,
    0, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(64),
    new ThreadPoolExecutor.AbortPolicy()
);

void afterPark(Ticket ticket) {
  notifyPool.execute(() -> sms.send(ticket));
}

// RejectedExecutionException → occupy already succeeded; retry notify later.

Name the rejection

The is not a default you shrug at. When the queue is full and every worker is busy:

  • Abort — throw. park() can catch and skip SMS, or fail the notify path. Occupancy can still stand.
  • CallerRuns — the gate thread sends SMS. Occupy waits on the network again. Sometimes that is the back-pressure you want. Often it is not.
  • Discard / DiscardOldest — drop work. Receipts vanish. Say so, or do not pick it for money-adjacent notes.

Factories that hide the ceiling

Executors.newCachedThreadPool() grows without bound. It is new Thread() with extra steps. newFixedThreadPool(n) with an unbounded LinkedBlockingQueue still has no real limit — the queue grows forever. Prefer constructing ThreadPoolExecutor so both worker count and queue capacity are visible.

java
// Hidden ceiling — do not hide this in a static singleton.
ExecutorService bad = Executors.newCachedThreadPool();

// Visible team. Shut it down when the process ends.
ThreadPoolExecutor ok = new ThreadPoolExecutor(
    2, 2, 0, TimeUnit.SECONDS,
    new ArrayBlockingQueue<>(64),
    new ThreadPoolExecutor.AbortPolicy()
);

when the process ends. shutdown(), then awaitTermination. In an LLD interview, a fixed pool of two gate workers, or a single allocator thread, is usually enough to show you thought about it.

Names, schedules, and stealing

Name threads in a ThreadFactory: notify-1, notify-2. Dumps and traces become readable. ScheduledExecutorService is a pool plus a delay wheel for “retry SMS in 5s” or a task scheduler’s due work. Same rules: bound the pool, bound what you schedule, cancel on shutdown. ForkJoinPool / work stealing is for CPU-split tasks, not for SMS. Do not reach for it in parking LLD.

java
ThreadFactory named = r -> {
  Thread t = new Thread(r, "notify-" + seq.incrementAndGet());
  t.setDaemon(true);
  return t;
};

ScheduledExecutorService retry =
    new ScheduledThreadPoolExecutor(1, named);
retry.schedule(() -> sms.send(ticket), 5, TimeUnit.SECONDS);

Daemon threads die when main exits; they will not keep a process alive to flush the last receipts. Non-daemon plus awaitTermination will. Pick based on whether leftover SMS matters. In a coding round, say you shut down and wait briefly.

Keep occupy off the pool

If park() only execute()s and returns, the caller has no Ticket. Either occupy on the caller (or allocator) and pool the side effect, or return a Future<Ticket> and make the caller wait on get(). The second design turns the pool into the occupancy team — then you must bound that team and explain LotFull under rejection. The first design is simpler and matches “two gates, one spot” on the request path.

Common mistakes

new Thread() per request

Threads are not free. A burst of park() calls will create more workers than the machine can run. Submit to a pool with a ceiling.

A pool with no ceiling

newCachedThreadPool() grows without bound. So does a queue of Integer.MAX_VALUE. Bound workers and the queue, then name what happens at the limit.

Running occupy on the pool

If park() only submits and returns, the caller has no ticket yet. Keep the occupancy decision on the request thread (or the allocator). Pool the side work.

Saturating Saturday, on a napkin

2 notify workers, queue 64, Abort. 200 cars in a minute, occupy is fast, SMS is 200ms. Two workers do about 10 SMS/s. The queue fills. Abort starts firing. Occupancy still succeeds if afterPark catches rejection. Drivers have tickets; some receipts lag or drop. CallerRuns would make gates send SMS and occupy would slow to the provider. DiscardOldest would lose the earliest receipts — usually the wrong ones. Draw that napkin. The number 2 and the number 64 are now a story, not hope.

CPU matching in a game or chess engine: size near cores, short tasks, no blocking inside the worker. Mixing SMS into that pool is how a core sleeps on I/O while a hash waits. Two pools, two stories.

Exercises

  1. 4 cores, SMS I/O, 200 cars in a burst. Sketch workers, queue size, and Abort versus CallerRuns for the 201st notify.
  2. Why is newFixedThreadPool(2) still dangerous if you never pass a bounded queue?
  3. Task scheduler: one pool for CPU matching, one for I/O dispatch. Why split them instead of one pool of 32?
  4. A static Executors.newCachedThreadPool() inside ParkingLot. What ceiling is missing, and when do those threads die?

Write three numbers on the board: worker count, queue capacity, rejection name. If you cannot, you have not sized the team. “A reasonable pool” is not a size.

Keep occupy on the caller unless you are ready to return Future<Ticket> and define LotFull under Abort. Side work belongs on the pool. Occupancy belongs on the gate or the allocator.

Read next

Where the pool still fails

Tasks that block on a lock the caller also needs: deadlock via the pool. Tasks that catch Throwable and swallow: silent death of a worker until the pool looks idle but does nothing. Unbounded schedule() of retries: a hidden unbounded queue. corePoolSize 0 with a filled queue and allowCoreThreadTimeOut can surprise you with no workers. Keep core equal to max for a tiny notify pool so the team is always there.

A shared static pool across ParkingLot tests leaks threads between cases. Inject the Executor. In production, one named pool per workload. Metrics: active count, queue size, rejected count. Without rejected count, Abort looks like a random exception at the gate.

  • Never submit park() itself if the caller needs the Ticket now.
  • Never mix CPU-hash and SMS on one pool of 32.
  • awaitTermination with a budget; then shutdownNow if you must, and say what you cancelled.

Takeaway

Size workers from the workload. Bound the queue. Name the rejection policy. Keep occupy on the caller. Do not hide newCachedThreadPool() in a singleton. Shut the pool down. A small visible pool beats two hundred anonymous stacks.

Three numbers: workers, queue, rejection. Name the threads. Split CPU work from SMS. Inject the Executor in tests. Count rejects. Daemon versus awaitTermination is whether leftover receipts matter when main exits.

newCachedThreadPool() and an unbounded LinkedBlockingQueue are the same missing ceiling. Visible ThreadPoolExecutor construction is the interview artifact. Two, sixty-four, Abort — say those out loud before you type execute().

Do not run occupy on the pool unless the caller waits on a Future and you have defined LotFull under rejection.

newCachedThreadPool() and an unbounded LinkedBlockingQueue are the same missing ceiling. Visible ThreadPoolExecutor construction is the interview artifact. Two, sixty-four, Abort — say those out loud before you type execute().