Cheat Sheet — Everything, Dense
This is the reference. One page per domain, nothing you cannot use in a room. Every formula, every number, every decision rule, every complexity, every failure mode, and the sentences to actually say out loud.
Read it end to end once. After that, open it the morning of a round and read only the section for that round. Everything here is taught from first principles in the WARMUP guides — the links are per section.
Table of Contents
- 1. The Loop, Round by Round
- 2. Coding — Representation First
- 3. Python Internals
- 4. Distributed Systems
- 5. Inference Infrastructure
- 6. Take-Home and Deep Dive
- 7. Behavioral
- 8. Agentic Coding
- 9. The Scripts
- 10. Pre-Round Checklist
1. The Loop, Round by Round
| Round | Reported shape | What it actually tests | The tactic |
|---|---|---|---|
| Recruiter | Background + "where is AI headed" | Whether you have a position | A falsifiable claim + a falsifier. Read the charter in a browser first |
| Screen A — coding | 60 min, CoderPad, versioned KV store | Representation choice under a clock | Ask the global-vs-per-key question in the first 90 seconds |
| Screen B — design | 60 min, Excalidraw, fault-tolerant job scheduler | Whether you find the load-bearing parts | Ask at-least-once vs at-most-once. Name the two hard parts at minute 10 |
| Take-home | 48h, "build something real" | Judgement under ambiguity | Decision log from hour zero. Walking skeleton by hour 8 |
| Deep dive | Line-by-line, from a list written after reading your code | Whether you decided or defaulted | Defend every constant. Volunteer the riskiest line |
| Coding 1 | Progressive, ~4 gates, pass bar reportedly 2 (assume 3) | Representation that survives unseen requirements | Time-to-first-gate ≤ 8 min. Never rewrite |
| Coding 2 | Systems-flavored: state, concurrency, memory | Justifying your own choices | Internals arrive as follow-ups to your code |
| Design — "design ChatGPT" | GPU allocation, autoscaling, coordination | Scoping judgement | Abstract the engine by default; open it in seconds when asked |
| Behavioral | Cross-team architecture, consensus under pressure | Staff scope, concrete tradeoffs | DTAO. Decision in sentence one. A named opponent |
| Agentic (beta) | Multi-file repo, oversized task, drive an agent | Reviewing a fast confident collaborator | Plan → checkpoints → reject one diff specifically |
Calibrate to Staff, not Senior — AI-lab levelling is compressed; the "Senior" title reportedly carries Staff scope. Ask the AI-tool policy per company, per round — policies are opposite at labs you may interview at in the same month.
2. Coding — Representation First
Taught in: tracks/coding/WARMUP.md
2.1 The query-shape table
The single most useful table in the whole document. Hear the query shape, not the noun.
| Query shape | Structure |
|---|---|
| "Is X present?" / "value at exactly K" | hash map |
| "largest key ≤ X" — predecessor / floor / as of | sorted array + bisect · balanced tree · skip list |
| "all keys in [A,B]" — range | sorted array · B-tree · LSM |
| "smallest element", repeatedly | heap |
| "least recently used" | intrusive doubly-linked list + hash map |
| "how many in the last N seconds" | deque you trim · ring buffer |
| "seen before, approximately" | Bloom / cuckoo filter |
| "what depends on what" | DAG + topological order |
| "prefix match" | trie |
A hash map answers exactly, never nearest. The moment a requirement says before, after, as of, range, nearest, at most — you need an ordered structure.
2.2 The three questions
Ask out loud in the first 90 seconds. Each has historically decided whether gate 3 was additive or a rewrite.
- "Is the input replayable / immutable?" → if yes, store positions into it, not copies. O(1) checkpoints instead of O(n).
- "Is this identifier global or per-entity?" → global makes a snapshot one integer. Per-entity forces a vector and every later gate is harder.
- "Will I ever need to undo this?" → if maybe, store deltas, not states. Deltas compose; snapshots do not.
2.3 The ten patterns, one line each
| # | Pattern | The load-bearing idea |
|---|---|---|
| 1 | Versioned KV / MVCC | As of V is a predecessor query. Per-key append-only (version, value) list + bisect. Delete is a tombstone. Versions are global. Snapshot = one integer. Compact = reachability from pins |
| 2 | Delta log | Store (cursor_before, n_events, token) per step. Checkpoint = three list lengths. Undo = pop. Redo cleared by a new edit |
| 3 | Intrusive list (LRU) | dict → node; node in a doubly-linked list. Sentinels kill every edge case. Lazy + sampled expiry. Evict until under budget |
| 4 | Rate limiting | Fixed window admits 2× at a boundary. Log is exact, O(limit)/key. Counter interpolates, O(1), ~1% error. Bucket separates rate (sustained) from capacity (burst) |
| 5 | Heap scheduling | Push (fire_at, priority, seq, id) — the seq is not optional. Cancel lazily + rebuild at 50% tombstones. Fixed-rate vs fixed-delay + a catch-up policy |
| 6 | Streaming state machine | A carry buffer + a state that survives the chunk boundary. A regex cannot say "no match yet" |
| 7 | Dependency graph | Three colours: WHITE/GREY/BLACK. GREY = back edge = cycle. Two states conflates "on my path" with "done". Reverse graph for incremental recompute |
| 8 | WAL | [len][payload][CRC]. A torn tail is expected, not an error. Checkpoint = temp → fsync → os.replace → fsync the directory |
| 9 | Dedupe | Exactly-once delivery is impossible. At-least-once + idempotent consumer. Bloom errors point the dangerous way for dedupe → use it as a negative cache in front of an exact store |
| 10 | Backpressure | An unbounded queue is a latency amplifier then an OOM. Four responses: block · buffer · shed · degrade |
2.4 Complexity table
| Structure | Lookup | Insert | Delete | Min/Max | Ordered scan |
|---|---|---|---|---|---|
| Hash map | O(1) | O(1) | O(1) | O(n) | impossible |
| Sorted array | O(log n) | O(n) | O(n) | O(1) | O(k) |
| Balanced BST / skip list | O(log n) | O(log n) | O(log n) | O(log n) | O(k) |
| Binary heap | O(n) | O(log n) | O(log n) root | O(1) | no |
| Doubly-linked list | O(n) | O(1) given node | O(1) given node | O(1) ends | O(n) |
| LRU (map + list) | O(1) | O(1) | O(1) | O(1) LRU | no |
| Trie | O(len) | O(len) | O(len) | — | prefix O(k) |
| Bloom filter | O(k) | O(k) | impossible | — | no |
| B-tree | O(log n) | O(log n) | O(log n) | O(log n) | O(k) |
| LSM tree | O(log n)×levels | O(1) amort | O(1) tombstone | — | O(k) merge |
Four sentences worth memorizing: hash maps answer exactly, never nearest · heaps keep only the partial order you need · O(1) removal needs the node and double links · append-only data is sorted for free when the key is monotonic.
2.5 The narration script
- Restate the problem in your own words.
- Clarify — ≥2 questions, ≥1 that could change your representation.
- State the approach in two sentences + name the data structure.
- State the complexity before implementing.
- Write the test for the tricky invariant first.
- Code, narrating decisions — not keystrokes. "I'm recording the event count per feed so I can unwind it later" is a decision. "Now a for loop" is not.
- When stuck, keep talking. Silence is the costliest narration failure.
2.6 Coding failure modes
| Symptom | Actual gap | Fix |
|---|---|---|
| Time-to-gate-1 > 16 min | Over-designing | 10-minute alarm; gate-1 sprints |
| Fast G1, rewrite at G3 | Bad representation | Better clarifying questions, not longer design |
| Passes tests, can't state complexity | Never says it out loud | State it before coding, always |
| Many test runs on the last gate | Testing at the end | Invariant-first |
| Narration ≤2 | Goes silent when stuck | Narration-only drill |
| Cold re-run much slower | Memorized, didn't learn | More variety, less repetition |
3. Python Internals
Taught in: tracks/python-internals/WARMUP.md · QUIZBANK.md
3.1 Version-dependent facts
Say the shape, then check — never recite a constant.
| Fact | Current state |
|---|---|
| Free-threading | PEP 703 designed it; PEP 779 defined "supported". Phase II in 3.14 (Oct 2025): officially supported, not default. Phase III unscheduled. sys._is_gil_enabled() |
| Free-threaded cost | ~5–10% single-thread overhead (was ~40% at 3.13), ~15–20% more memory, ~4× on suitable multi-threaded CPU work |
| gc thresholds | (gen0, gen1, gen2). Long-documented (700,10,10); (2000,10,10) on 3.13, which also added an incremental collector. Say gc.get_threshold() |
CancelledError | Inherits BaseException since 3.8 |
TaskGroup / except* | 3.11+ |
bisect(key=...) | 3.10+ |
| PEP 479 | StopIteration escaping a generator → RuntimeError, default since 3.7 |
| PEP 442 | __del__ in cycles collectable since 3.4 |
| PEP 412 | Key-sharing dicts — narrows the __slots__ win, so measure |
| Managed dict | 3.11+ — instance dict is lazy, so a non-slotted subclass costs nothing until you store in it, then ~5.8× |
3.2 Generators
| Thing | Answer |
|---|---|
| Protocol | __iter__ + __next__, ends with StopIteration |
| Iterable vs iterator | Iterable returns a new iterator; iterator returns self and holds position |
| Calling a generator function | Returns a generator object. Runs nothing |
| What it holds | A suspended frame: locals + instruction pointer + eval stack |
send on a fresh generator | TypeError — it's suspended before the first yield; prime with next() |
throw outcomes | (a) propagates + closes · (b) generator yields → throw returns it · (c) generator returns → StopIteration |
close() | Throws GeneratorExit at the yield → finally runs. Catching it and yielding again → RuntimeError |
yield from | Delegates iteration and forwards send/throw/close and captures the sub-generator's return value |
zip | Over-consumes — pulls one extra from every iterator before the shortest ends, and discards it |
tee | Buffers everything one branch read that the other hasn't. Draining one materializes the stream |
| Memory | 2M-item list ≈ 77 MiB; generator ≈ 400 bytes. Only cheaper if you never need it twice |
3.3 Async
| Thing | Answer |
|---|---|
| Event loop | A queue of ready callbacks + one blocking epoll/kqueue call. Single-threaded |
| Coroutine vs Task | Coroutine is inert. create_task schedules it concurrently. await runs it inline |
| Fire-and-forget | Loop holds only a weak ref — keep a strong one or use a TaskGroup |
gather on failure | Propagates the first exception, does NOT cancel siblings → orphans → resource leak |
TaskGroup | Cancels siblings, raises ExceptionGroup, caught with except*. Structured concurrency |
CancelledError | BaseException. except Exception correctly won't catch it. If you catch it, re-raise |
| Cancellation | Cooperative — delivered at a suspension point. A tight CPU loop can't be cancelled |
| Blocking call | Stalls the whole loop. Measured: a 10 ms ticker's max gap goes 10 ms → 162 ms |
| Escape hatch | await asyncio.to_thread(fn) for blocking I/O · process pool for CPU |
| Async generators | Cleanup must await, so it can't run in GC. Use contextlib.aclosing or leak connections |
| asyncio primitives | Not thread-safe. call_soon_threadsafe is the only cross-thread door |
3.4 Concurrency decision table
| Model | Wins on | The cost that makes it lose |
|---|---|---|
| asyncio | Thousands of concurrent I/O waits; high fan-out RPC | One blocking call stalls everything; async all the way down |
| Threads | Blocking I/O through non-async libs; moderate concurrency | No CPU parallelism under the GIL; ~8 MB stack each; shared-state bugs |
| Processes | CPU-bound work | Serialization per call; memory duplication; slow startup; no shared objects |
async for waiting, processes for computing, threads for when the library gives you no choice.
The GIL: guarantees one thread runs bytecode at a time, so single bytecodes and
non-releasing C calls are atomic. Does not make your sequences atomic. list.append atomic;
counter += 1 not.
The nuance that separates you: since 3.10 the eval breaker is checked only at specific
instructions (backward jumps, calls). So a bare counter += 1 loop often loses zero updates
— the check lands after the STORE. Put a call between load and store and it loses 3%; make
__add__ a Python method and it loses 61%. "I ran it and it didn't lose anything" is not
evidence of atomicity.
3.5 Memory
| Thing | Answer |
|---|---|
| Object header | ob_refcnt + ob_type. No primitives — an int is a heap object |
| Refcounting | Prompt and deterministic; can't collect cycles; every ref op touches memory |
| Cycle collector | Generational mark-and-sweep over containers only. Subtract internal refs; nonzero remainder = live |
__del__ in a cycle | Runs (PEP 442). Hazards: undefined order, exceptions swallowed, resurrection → prefer weakref.finalize / context managers |
| Allocator | Arenas (256 KB) → pools (4 KB, one size class) → blocks. ≤512 B via pymalloc |
| "Freed but RSS didn't drop" | Expected — an arena releases only when every pool in it is empty |
__slots__ | Removes the per-instance __dict__. Measured ~38% over 200k 3-attr instances |
__slots__ breaks | New attributes; weakrefs unless you add '__weakref__'; multiple inheritance from two slotted bases |
| Subclass trap | A subclass omitting __slots__ regains a __dict__. On 3.11+ it's lazy, so identical size until you store in it — then 5.8×. Tell = hasattr(x,'__dict__'), not size |
sys.getsizeof | Own footprint only. 50k strings: reports 434 KiB, costs 3,510 KiB. Use tracemalloc |
memoryview | Zero-copy. 16 MiB slice: bytes copies 16 MiB, view allocates ~0. A live view pins the bytearray |
| Interning | −5..256 cached. 257 folded within one code object. int("257") is int("257") → False. Never use is for values |
3.6 Data model
Attribute lookup order: data descriptor on type → instance __dict__ → non-data
descriptor → class attrs up the MRO → __getattr__.
| Thing | Answer |
|---|---|
| Data descriptor | Defines __set__/__delete__ too → outranks the instance dict → @property can't be shadowed |
| Non-data descriptor | Only __get__ → instance dict wins → methods can be monkeypatched |
Why self binds | A function is a non-data descriptor whose __get__ returns a bound method |
__getattribute__ | Every access. The perf hazard. Classic bug: self.__dict__[name] inside it → RecursionError. Delegate to object.__getattribute__ |
__getattr__ | Only on AttributeError fallback. Free on hits. Use for proxies/lazy |
super() | Not "the parent" — the next class in the MRO of type(self). In Diamond(Left,Right), Left's super reaches Right |
| MRO | C3 linearization. Inconsistent → TypeError at class definition |
__exit__ | Returning truthy suppresses the exception. A bare return True eats every bug |
__eq__ without __hash__ | Sets __hash__ = None → unhashable, deliberately |
3.7 The traps
| Trap | Why |
|---|---|
| Mutable default arg | Evaluated once at def, stored on the function object |
lru_cache on a method | Keys on self → holds a strong ref to every instance ever → unbounded leak |
lru_cache keying | f(1,2) and f(1,b=2) are different keys |
String += in a loop | Immutable → O(n²). Use "".join |
list.pop(0) | O(n). Use deque.popleft() |
fork with threads | Only the forking thread survives; a lock held elsewhere is held forever in the child |
if k not in d: d[k]=v | Not atomic. dict.setdefault is |
4. Distributed Systems
Taught in: tracks/systems-design/WARMUP.md
4.1 The arithmetic
Little's law \( L = \lambda W \) — items in system = arrival rate × time in system.
- Sizing: 50k rps × 8 ms = 400 concurrent.
- Latency from depth: 10,000 queued ÷ 100/s = 100 s wait.
- Pool: 500 qps × 20 ms = 10 connections busy.
The utilization knee — M/M/1: \( W = W_s / (1-\rho) \)
| ρ | 0.5 | 0.7 | 0.8 | 0.9 | 0.95 | 0.99 |
|---|---|---|---|---|---|---|
| × service time | 2.0 | 3.3 | 5.0 | 10 | 20 | 100 |
Latency is hyperbolic in utilization, not linear. Real traffic is burstier than Poisson, so the knee arrives earlier. This is the whole quantitative argument for admission control.
Retry amplification = \( \sum_{i=0}^{k-1} f^i \)
| Failure rate | 10% | 50% | 80% | 95% | 100% |
|---|---|---|---|---|---|
| 3 attempts | 1.11× | 1.75× | 2.44× | 2.85× | 3.00× |
Storage: 1M × 1 KB = 1 GB · 1B × 1 KB = 1 TB · day ≈ 10⁵ s · month ≈ 2.5×10⁶ s.
4.2 Latency numbers
| Operation | Time |
|---|---|
| L1 cache | 1 ns |
| Branch mispredict | 3 ns |
| L2 cache | 4 ns |
| Mutex lock/unlock | 17 ns |
| Main memory | 100 ns |
| Compress 1 KB (snappy) | 2 µs |
| Read 1 MB seq from memory | 3 µs |
| SSD random read | 16–100 µs |
| Read 1 MB seq from SSD | 49 µs |
| Round trip same DC | 500 µs |
| Read 1 MB seq from disk | 825 µs |
| Disk seek | 10 ms |
| RT US cross-country | 40–70 ms |
| RT US ↔ Europe | 80–150 ms |
Two derived rules: memory ~100× SSD, SSD ~100× disk seek · any cross-service hop ≥ 0.5 ms, so five sequential hops have a 2.5 ms floor → fan out, don't chain.
4.3 Leases and fencing
The highest-value 200 words in the track.
A lock held by a dead holder is held forever. A lease expires → liveness. But:
t=0 A takes a 30 s lease on J. Starts work.
t=10 A GC-pauses / partitions / is descheduled.
t=30 Lease expires. A hasn't renewed.
t=31 B takes the lease. Runs J.
t=45 B finishes, writes.
t=50 A wakes. From A's view NOTHING HAPPENED. Finishes J and writes.
A's stale write lands AFTER B's correct one.
You cannot detect this. Unreachable and dead are indistinguishable from outside — that is a theorem. And "check your lease before writing" fails too: the pause can land between the check and the write.
Fencing token — a monotonically increasing number issued with every lease grant. The resource rejects any write with a token below the highest it has seen.
UPDATE results SET value=%s, fence=%s
WHERE job_id=%s AND fence < %s; -- 0 rows = superseded. Do not retry.
Where it's checked matters more than the token. The resource must enforce it — not the lock service, not the client. A zombie client believes its token is current. If the resource can't participate (a third-party API with no conditional write), say so and mitigate with idempotency or accept at-most-once.
Lease sizing: renew at lease/3 so two missed heartbeats are tolerable. Too short → GC pauses cause spurious expiry and routine double execution. Too long → a dead holder blocks work. 60 s / 20 s is a reasonable default.
Redlock: relies on bounded clock drift and bounded pauses for correctness; neither is guaranteed. The safety argument lives in fencing, not the lock protocol.
4.4 Replication and quorums
| Mode | Ack when | On failover | Latency |
|---|---|---|---|
| Sync | all replicas | zero loss | slowest replica |
| Async | leader only | acked writes can be lost | fastest |
| Semi-sync | ≥k replicas | lose only if >k fail together | one slow replica tolerated |
Quorum: \( W + R > N \) forces overlap (pigeonhole) → a read sees the latest write.
| N | W | R | Property |
|---|---|---|---|
| 3 | 2 | 2 | Standard, tolerates 1 failure both ways |
| 3 | 3 | 1 | Fast reads, no write availability on any failure |
| 5 | 3 | 3 | Tolerates 2 failures |
| 3 | 1 | 1 | W+R=2 ≤ 3 → no overlap, eventual only |
What a quorum does NOT give you (name three): sloppy quorums break the overlap · concurrent writes still need conflict resolution · a write that fails after reaching some replicas isn't rolled back · read-your-writes isn't guaranteed across sessions · quorum reads aren't linearizable without read-repair-then-commit.
Convergence: read repair (cheap, on the read path, misses cold data) + anti-entropy (Merkle trees — compare root hashes, descend only into differences, O(log n) in the difference).
4.5 Consensus
What it buys: agreement on an ordered log → replicated state machine → anything.
FLP: in a fully asynchronous system with one faulty process, no deterministic algorithm guarantees consensus. Practical systems add timeouts → safety always, liveness under assumptions.
Raft:
- Terms are a logical clock. Higher term seen → step down. That one rule kills most split-brain.
- Randomized election timeouts (150–300 ms) break split votes without coordination.
- An entry is committed once on a majority.
- Log matching: same index + same term ⇒ identical logs up to that point (by induction).
- Safety rule 1 — election restriction: a voter refuses a candidate whose log is less up-to-date. Two majorities must intersect ⇒ any winner has every committed entry.
- Safety rule 2 — never commit a previous term's entry by counting replicas. A leader commits only its own term's entries by counting; older ones commit indirectly. In practice a leader appends a no-op to trigger it. (This is the part people skip.)
Costs — say these: every write is ≥1 RT to a majority (~1 ms same-DC, 50–150 ms cross-region) · the leader is a throughput ceiling → shard into many Raft groups · failover is a latency spike · odd numbers only (3→4 doesn't improve tolerance) · membership change is the hard part.
Use consensus for metadata, not data.
4.6 Consistency models
| Linearizability | Serializability | |
|---|---|---|
| About | single objects | multi-object transactions |
| Guarantees | recency (real-time order) | isolation (equivalent to some serial order) |
| Silent about | transactions | real time |
Strict serializability = both. That's Spanner, and why it needs TrueTime.
Snapshot isolation permits write skew — two txns read overlapping data, write disjoint
keys, both commit, jointly break an invariant. Canonical: on-call doctors. Fixes: SELECT ... FOR UPDATE · materialize the conflict · SSI (Postgres SERIALIZABLE).
Session guarantees (cheap, and what users notice): read-your-writes · monotonic reads · consistent prefix · causal · bounded staleness.
CAP, precisely: when a partition occurs, choose consistency or availability. Only during a partition. "Available" means every non-failing node responds. Say PACELC: if P then A or C, Else Latency or Consistency.
4.7 Time
| Clock | Use for | Never |
|---|---|---|
Wall (time.time) | timestamps | measuring durations — NTP steps it |
Monotonic (time.monotonic) | durations, leases | comparing across machines |
NTP holds a few ms on a good LAN — a statistical claim, not a bound. A node cannot know its own skew.
| Mechanism | Gives | Cannot |
|---|---|---|
| Lamport | total order consistent with causality | detect concurrency |
| Vector clocks | detects concurrency (conflicts) | O(nodes) size; pruning is subtle |
| HLC | close to physical + respects causality + O(1) | detect concurrency |
| TrueTime | bounded uncertainty interval | be cheap — commit-wait pays it in latency |
Last-write-wins by wall clock is a documented data-loss mode when a node's clock is skewed.
4.8 Partitioning
| Hash | Range | |
|---|---|---|
| Distribution | even | uneven, hot spots easy |
| Range queries | impossible | efficient |
| Resize | needs consistent hashing | split ranges |
hash(key) % N → changing N moves ~80% of keys. Consistent hashing → only K/N move,
and only from one neighbour. Virtual nodes (100–256/node) fix two things: load variance
(drops as 1/√v) and — more important — a node's failure spreads across many successors instead
of dumping its whole range on one and cascading.
Hot partitions: cache it (usually the whole answer) → split the key (id:0..99) → dedicated
partition → rate-limit it. Per-key metrics come first — you can't fix what you can't see, and
aggregates hide it completely.
Rebalancing: snapshot → stream delta → briefly block + flip ownership. Fence the flip. Never auto-rebalance on failure (a blip triggers a storm). Rate-limit the copy.
4.9 Delivery semantics
Exactly-once delivery is impossible — sender can't distinguish "never arrived" from "arrived, ack lost"; more round trips just move it to the ack of the ack (Two Generals).
Exactly-once processing = at-least-once + idempotent consumer. Requires a stable idempotency key generated by the PRODUCER, unchanged across retries. Generate it at send time and dedupe silently does nothing.
Dual write — db.save(x); queue.publish(x) — has no safe ordering. Fix: outbox —
insert into an outbox table in the same transaction, a relay publishes with FOR UPDATE SKIP LOCKED. Or CDC (tail the WAL/binlog).
DLQ needs four things, and most designs mention only the first: the reason (error, stack, attempt count) · a replay path · poison detection (permanent vs transient — don't retry 4xx) · alert on arrival rate, not depth. And state the ordering consequence: if message 5 dead-letters, 6 either blocks (order preserved) or proceeds (order broken). No third option.
4.10 Load control
Fix order — most people get this backwards:
- Retry budget — cap retries at ~10% of base traffic. Bounds amplification at 1.1× no matter how bad it gets. (gRPC retry throttling, Envoy retry budgets.)
- Circuit breaker — stop trying entirely.
- Jitter — desynchronize what you do send.
Jitter alone still delivers 2.85× at a 95% failure rate. And don't retry at every layer — 3 layers × 3 retries = 27 attempts.
Backoff variants: full uniform(0, min(cap, base·2^n)) — AWS's simulation found it minimized
both total work and completion time · equal temp/2 + uniform(0, temp/2) — keeps a floor ·
decorrelated uniform(base, prev·3) — smoothest, hardest to bound.
Circuit breaker: closed → open (fail immediately) → half-open (one probe, not a flood). Threshold must be a rate over a minimum volume ("50% over ≥20 requests in 10 s") — an absolute count breaks on low-traffic endpoints. Scope per endpoint, often per instance.
Shedding: rejecting 10% in 1 ms beats accepting 100% and timing all out at 30 s. Shed by priority, not at random. Shed the OLDEST queued item — under sustained overload FIFO serves only requests whose clients have given up. Deadline propagation: pass the remaining budget downstream; fail fast if there isn't enough left.
Bulkheads: separate pools per dependency. Honest counter-argument: by M/M/c, one pool of 50 has better tails than five of 10 — you trade efficiency for isolation. Cellular architecture is the strongest blast-radius answer: 1/N by construction.
4.11 The failure catalog
| Failure | Detection | Containment | Recovery |
|---|---|---|---|
| Node crash | heartbeat / lease expiry | traffic drains | replacement joins; re-replicate |
| Fail-slow | latency percentiles vs peers, outlier detection | eject on latency SLO, timeouts everywhere | restart. Never trust its self-report |
| Partition | quorum loss on the minority | minority refuses writes | merge + reconcile on heal |
| Zombie holder | undetectable | fencing token rejected at storage | nothing to recover |
| Thundering herd | queue depth spike | rate-limited catch-up, jittered restarts | bounded drain |
| Retry storm | rate up while success down | retry budget | circuit break → half-open probe |
| Poison message | attempt count | DLQ after N | replay after fix |
| Hot partition | per-key metrics | split / cache / limit | rebalance |
| Cascading failure | correlated latency | bulkheads, timeouts, shedding | shed → ramp |
| Corruption | checksums, invariant audits | quarantine, stop replicating | restore to known-good |
| Clock skew | skew monitoring | treat as unhealthy | resync; re-elect |
| Bad config rollout | canary divergence | staged rollout | auto-rollback |
Fail-slow is worse than fail-stop and it's the common case — the node answers your health check in 2 ms while serving real requests in 40 s. If your only health signal is "does it respond", you haven't handled the common case.
Say one deliberately accepted failure mode with its cost. That's a staff move; claiming to have handled everything is falsifiable in one question.
4.12 The design template and clock
1 Requirements & scope (incl. explicitly out of scope)
2 Scale numbers (and the arithmetic you did)
3 API surface (3-5 calls that matter)
4 Data model (keys and indexes, and WHY)
5 High-level arch (the diagram)
6 DEEP DIVE: the two hardest components
7 Failure & recovery (detection · containment · recovery, each)
8 Bottlenecks & evolution
9 Tradeoffs explicitly rejected
| Min | Do |
|---|---|
| 0–5 | Clarify. Scale numbers written down |
| 5–10 | API + data model |
| 10–20 | Architecture + diagram |
| 20–35 | Deep dive on the two hard parts |
| 35–45 | Failure, bottlenecks, rejections |
Finding the two hard parts: where must ≥2 machines agree? · where can data be lost? · what's highest-rate/cardinality? · where does one tenant affect another?
Say at minute 10: "I think the two places this can actually fail are X and Y, so that's where I want to spend the time — does that match what you care about?"
5. Inference Infrastructure
Taught in: tracks/ml-infra/WARMUP.md · runnable: gpu_math.py
5.1 The one derivation
Arithmetic intensity \( I = \text{FLOPs} / \text{bytes moved} \). Machine balance = peak dense FLOP/s ÷ bandwidth. H100: \( 989.5 \times 10^{12} / 3.35 \times 10^{12} \approx \mathbf{295}\) FLOP per byte.
Decode: every weight is read once per step regardless of batch; each sequence does ~2N FLOPs.
\[ I_{\text{decode}} \approx \frac{2Nb}{2N} = b \qquad\text{(the batch size)} \]
At batch 1 you use ~1/295 of the GPU's compute. Say "dense". The datasheet's 1,979 TFLOP/s is with 2:4 sparsity; LLM weights are dense, so the real number is 989.5 and the balance is 295, not 590. Quoting the sparse figure for a dense workload is a cheap way to look like you read a spec sheet instead of a benchmark.
70B FP16 on H100:
weights 140 GB / 3.35 TB/s = 41.8 ms (memory)
2 x 70e9 / 989.5e12 = 0.14 ms (compute, dense BF16)
ratio 295x → MEMORY-BOUND (exactly the machine balance — a good self-check)
Prefill: all s prompt tokens at once → \( I \approx s \). A 2,000-token prompt = 280 TFLOP = 141 ms of solid compute → compute-bound, and it blocks everyone's decode.
Prefill is compute-bound, decode is memory-bound. Two workloads on one accelerator. That sentence is most of this round.
The proof: H200 has identical compute to H100 (989.5 TFLOP/s BF16 dense) and +43% bandwidth (4.8 vs 3.35 TB/s), and is materially faster at decode. If decode were compute-bound it would be exactly as fast.
5.2 Memory formulas
| Quantity | Formula |
|---|---|
| Weights | N × bytes/param |
| KV per token | 2 × layers × kv_heads × head_dim × bytes |
| Max batch | (HBM − weights − activations) / (kv_per_token × seq_len) |
| Decode step floor | (weight_bytes + kv_bytes) / bandwidth |
| Decode intensity | ≈ batch |
| Prefill intensity | ≈ prompt_len |
| Prefill FLOPs | ≈ 2 × N × prompt_tokens |
| Decode FLOPs/token | ≈ 2 × N |
| Precision | B/param | 70B |
|---|---|---|
| FP32 | 4 | 280 GB |
| FP16/BF16 | 2 | 140 GB |
| FP8/INT8 | 1 | 70 GB |
| INT4 | 0.5 | 35 GB |
5.3 Hardware numbers
| GPU | Memory | Bandwidth | BF16 | FP8 |
|---|---|---|---|---|
| A100 80GB | 80 GB HBM2e | 2.04 TB/s | 312 | — |
| H100 SXM | 80 GB HBM3 | 3.35 TB/s | 989.5 | 1,979 |
| H200 SXM | 141 GB HBM3e | 4.8 TB/s | 989.5 — same | 1,979 |
| B200 | 192 GB HBM3e | ~8 TB/s | ~4,500 | ~9,000 (FP4) |
Cloud $/hr, 2026-reported, order of magnitude: H100 ~$1.50–3.00 · H200 ~$3.80 · B200 ~$6.50. Always attach a date to a price.
5.4 Llama-70B anchors
| Weights FP16 | 140 GB → needs ≥2 H100s, 4 in practice |
| KV/token (L=80, kv_heads=8, d_h=128) | 320 KB |
| KV @ 4k context | 1.25 GB/sequence |
| KV @ 128k context | 39 GB — half an H100 for one user |
| GQA saving vs MHA | 8× (2.5 MB/token → 320 KB) |
| 4×H100 available for KV | 320 − 140 − 16 = 164 GB |
| Max batch @4k | ~130 |
| Decode step @ that batch | 304 GB read → 91 ms |
| Aggregate | ~1,440 tok/s |
| Per user | ~11 tok/s ≈ reading speed (sanity check!) |
| Cost/1M output tokens | ~$5 (4×H100 @ $2.50/hr, 40% realized) |
At batch 130 the KV cache (164 GB) exceeds the weights (140 GB). Doubling context halves concurrency, one for one.
5.5 Technique tradeoffs
| Technique | Buys | Costs | Loses when |
|---|---|---|---|
| Continuous batching (Orca) | no idle slots, no HOL blocking | scheduler complexity, TPOT jitter | ~never — table stakes |
| PagedAttention (vLLM) | near-zero KV fragmentation → bigger batch; CoW prefix sharing | indirection per attention op, custom kernel | ~never |
| Chunked prefill (Sarathi) | much better TTFT/TPOT tails | slightly lower prefill throughput | throughput > tails |
| Prefix caching | skips prefill for shared prefixes; O(n²)→O(n) in a chat | cache memory competes with KV; eviction policy; must be tenant-scoped or it leaks | prefixes aren't shared |
| Speculative decoding | 1.5–3× latency at low batch; output distribution provably identical | wasted compute on rejects; a second model | high batch — no spare compute |
| Quantization | halves the dominant bytes → helps decode twice | quality, workload-specific | quality is the product |
| Disaggregated P/D | each scales independently | KV transfer over the network | transfer > interference avoided |
Parallelism: TP splits each layer — all-reduce every layer, needs NVLink, within a node, improves latency. PP splits layers — one transfer per boundary, across nodes, bubbles, does not improve latency. EP for MoE — all-to-all per layer, expert load imbalance is the hot-partition problem again.
The framing that scores: these aren't a stack of free wins, they're points on a throughput-vs-tail-latency curve — and it isn't one curve. Interactive chat, agentic loops, and batch want different scheduler policies → separate pools or one priority-aware scheduler with preemption. (Recompute is usually cheaper than swapping KV over PCIe.)
5.6 Autoscaling signals
Request cost varies ~10,000× ("hi" ≈ 1 unit; a 100k-token agent step ≈ 50,000).
| Signal | Verdict |
|---|---|
| Requests/sec | ✗ nearly uncorrelated with load |
| GPU utilization | ✗ misleading — high while memory-stalled |
| Queue depth / wait | ✓ direct measure of unmet demand |
| Tokens/s (prefill and decode separately) | ✓ the real unit of work |
| KV cache occupancy | ✓ best leading indicator — the binding constraint, rises before queueing |
| TTFT/TPOT p95 | ✓ good SLO trigger |
Composite: max(kv_occupancy/0.85, queue_wait_p95/target).
Scale-up is minutes (instance acquisition + pulling + loading 140 GB of weights + CUDA graph capture) → reactive is always late. Forecast from the diurnal curve · warm pools sized by forecast error, not average load · admission control as the fast path.
Fairness on TOKENS, not requests — ten 100k-token requests is 1,000× ten small ones.
5.7 The two altitudes
Default (abstracted) — say this verbatim:
"I'll treat the inference engine as a service with three properties: it exposes capacity in tokens per second rather than requests per second, it has an admission interface I can apply backpressure to, and it streams. I'll spend my time on traffic, coordination and failure — tell me if you want me to open it up."
When they say "open it up", lead with the constraint, not a build-up:
"The binding constraint is memory bandwidth, and the binding capacity is the KV cache."
Then: memory budget → why decode is bandwidth-bound (+ H100/H200 proof) → therefore batching → continuous batching + PagedAttention → the prefill/decode conflict → chunked prefill → prefix caching → the curve framing.
Never claim knowledge of their internal stack. Say: "I don't know how you do this internally, so I'll reason from the public systems — PagedAttention, Orca's continuous batching, Sarathi's chunked prefill — and you can tell me where that diverges."
6. Take-Home and Deep Dive
Taught in: tracks/take-home/WARMUP.md
The take-home and the deep dive are ONE round. The take-home generates a personalized interrogation surface. Every decision in the 48 hours is a question in week three. So the target isn't "best code" — it's code every line of which I can defend, plus a written record of the alternatives I rejected.
| Hours | Phase |
|---|---|
| 0–2 | Interrogate the brief → written list of ambiguities + decisions |
| 2–4 | Design doc v1 |
| 4–8 | Walking skeleton, committed, green |
| 8–28 | Implementation with tests as you go |
| 28–34 | Sleep. Non-negotiable |
| 34–40 | The hard part you deferred |
| 40–44 | One benchmark + methodology |
| 44–47 | README, design doc v2, commit history |
| 47–48 | Buffer |
Never cut: tests + a one-command runner · README that works on a clean machine · design doc
with tradeoffs · ambiguities section · decisions.md · error handling on external
boundaries · clean commit history · one benchmark · "what I'd do with two more days".
"Beyond the ask" = ONE of: a measured benchmark with honest methodology (including the disappointing number) · a failure-injection test that proves a recovery path · an operational concern nobody asked for (structured logs + correlation ID, real health check, DLQ runbook). Not more features — extra features read as poor judgement.
Cut order: extra features → configuration breadth → non-benchmark optimization → admin/UI → persistence sophistication. Write down what you cut and why — a documented cut is a decision.
Decision log entry — five fields:
Decision · Alternatives (with numeric reasons) · Assumes · Would revisit if · Not tested
The last two are what make it interview-grade. Write one for every constant.
The seven interrogation classes: Choice ("why X not Y") · Magic number ("why 30 s") · Scale ("at 100×") · Data loss ("where can this lose a message") · Omission ("what didn't you test") · Regret ("what would you change") · Hostile ("this function does four things").
The closing move — unprompted: name the riskiest line in your own diff and the next thing you'd build. Volunteering your design's weakest point before it's found is the single most credibility-generating move in the round.
7. Behavioral
Taught in: tracks/behavioral/WARMUP.md
DTAO, not STAR — STAR buries the decision.
| Section | Share | |
|---|---|---|
| D | Decision, one sentence, first | 1 sentence |
| T | Tradeoff — alternatives + why each lost. Numbers | ~40% |
| A | Alignment — who disagreed, what you did | ~30% |
| O | Outcome — measured, including what you got wrong | ~25% |
Context goes in a clause: "On the multilingual ranking pipeline, we decided X."
Four sentences that make it Staff:
- "The decision was ___." (first)
- "I rejected ___ because ___." (with a number)
- "___ disagreed, and their argument was ___." (stated fairly)
- "I got ___ wrong." (specific, generalizable lesson)
The test: if nobody disagreed, it isn't a Staff story.
Twelve required categories: cross-team architecture · a disagreement you lost · one you won · an outage you owned · a project you killed · raising a team's bar · a bet that failed · consensus without authority · deadline vs quality · changed your mind from data · non-engineers · what you'd build differently.
The disagreement-you-lost failure modes: humble-brag ("six months later they did it my way") · victim ("political reasons") · trivial · revisionist ("they were right about everything").
The seven probes — and the first is the discriminating one:
- "What was their strongest argument?" ← if you can't produce one, the story is suspect
- "What would have made the other option win?"
- "Who else was affected that you didn't mention?"
- "What did that cost the other team?"
- "How much of the timeline was the disagreement?"
- "How do you know it wasn't a coincidence?"
- "What would you do differently?" ← never answer "communicate more"
Never: lead with situation · a story with no disagreement · strawman the opposition · say "we" throughout · present single-team as cross-team · perform humility or enthusiasm · over-rehearse (recited is audible).
8. Agentic Coding
Taught in: tracks/agentic/WARMUP.md
The eight steps: baseline first (pytest, record count + runtime) → read ≤10 min → write
the plan (end state, 3–5 verifiable checkpoints, the invariant, what you'll do manually) →
delegate in checkpoint units → verify every checkpoint, read the diff even when green →
reject specifically → take over after two failed attempts on the same checkpoint →
narrate continuously.
Prompt properties: scope ("do not modify other files") · contract (exact signatures + validation) · invariant ("all N existing tests pass; behaviour with X=None byte-identical") · verification ("add tests/test_x.py covering…") · domain knowledge it lacks ("close the response body before retrying or we leak connections") · negative space ("do not catch CancelledError — it's a BaseException").
Never re-prompt with "that didn't work." Say what failed, why it's wrong at a domain level, and what to do instead.
Take over when: two failures on one checkpoint · the invariant isn't expressible as a test · you need to understand it to defend it later · a plausible-looking wrong answer is worse than slow progress. Don't take over the 30-file rename — that's the inverse error.
A smaller verified diff beats a larger unverified one. One specific rejection is worth more than three accepted diffs.
9. The Scripts
Verbatim sentences worth having ready.
Coding — clarifying (pick the one that changes your representation):
"Are versions global or per key?" · "Is the source replayable and deterministic?" · "Will I need to undo this?" · "Do you want events returned per call or accumulated?"
Design — scoping, at minute 10:
"I think the two places this can actually fail are X and Y, so that's where I want to spend the time — does that match what you care about?"
Design — at-least-once, obligated follow-through:
"That selects at-least-once — which means job handlers must be idempotent, and I'll give each execution a stable idempotency key so they can be. Exactly-once execution of a side-effecting job isn't achievable without cooperation from the job itself, so I won't claim it."
Design — the zombie:
"I can't distinguish a dead worker from an unreachable one — that's a theorem, not a gap in my design. So I'm choosing at-least-once and making the duplicate safe with a fencing token checked at the storage layer."
Design — accepting a failure mode:
"If we lose a region mid-write, in-flight requests are lost. I'm accepting that: synchronous cross-region replication adds 80 ms to every write, which blows the 200 ms p99 for the 99.99% of the time there's no regional failure. I replicate async and expose an RPO of ~5 seconds. If the business needs RPO zero, that's a different design and a different latency budget."
Inference — the abstraction:
"I'll treat the inference engine as a service with three properties: capacity in tokens per second, an admission interface I can apply backpressure to, and streaming. I'll spend my time on traffic, coordination and failure — tell me if you want me to open it up."
Inference — the boundary of your knowledge:
"I don't know how you do this internally, so I'll reason from the public systems — PagedAttention, Orca's continuous batching, Sarathi's chunked prefill — and you can tell me where that diverges."
Deep dive — the closing move:
"Two things you haven't asked about. The riskiest thing in here is ___ — it's correct but it's where a future change silently breaks ___, and it's the least-covered path. And the thing I'd build next isn't a feature, it's ___."
Behavioral — a number you can't defend:
"I'll be honest that it's a judgement call rather than a measurement — if I had production data I'd set it from the observed distribution."
Any round — being wrong:
"You're right. (pause) The reason I did it that way was ___, which is a weak reason. What I'd do instead is ___."
Any round — a performance claim:
Either "I measured 2.8× on my harness at batch 32 with a 512-token prompt" or "vLLM's published benchmarks report 3–5×". Never assert an unattributed number.
10. Pre-Round Checklist
The week before
- ☐ Ask the recruiter the AI-tool policy, per round — policies are opposite at different labs
- ☐ Confirm the tooling (CoderPad? Excalidraw? their environment?) and practise in it
- ☐ Read the charter/core-views in a browser; fill the
[VERIFY]slots - ☐ Read their two most recent engineering posts; write one specific question about each
- ☐ Two consecutive full-loop sims at hire (staff) or better
- ☐
review/queue clear, no leeches - ☐ All six forward-looking answers rehearsed within 7 days
The morning of
- ☐ Read only this file's section for that round
- ☐ Timer visible · recorder on for practice, off for the real thing
- ☐ Water. Eat. The screen is two 60-minute rounds back to back and fatigue is measurable
In the first two minutes of any technical round
- ☐ Restate the problem
- ☐ Ask the question that could change your approach
- ☐ State scale assumptions out loud, with numbers
- ☐ Name what's out of scope
In the last two minutes
- ☐ State what you'd do next and why you didn't
- ☐ Name the weakest part of what you built
- ☐ Ask one of your three prepared questions
Every claim here is taught from first principles in the WARMUP guides, and every number is
verified — see STATE.md for what was measured and when.