Cheat Sheet — Everything, Dense

This is the reference. One page per domain, nothing you cannot use in a room. Every formula, every number, every decision rule, every complexity, every failure mode, and the sentences to actually say out loud.

Read it end to end once. After that, open it the morning of a round and read only the section for that round. Everything here is taught from first principles in the WARMUP guides — the links are per section.


Table of Contents


1. The Loop, Round by Round

RoundReported shapeWhat it actually testsThe tactic
RecruiterBackground + "where is AI headed"Whether you have a positionA falsifiable claim + a falsifier. Read the charter in a browser first
Screen A — coding60 min, CoderPad, versioned KV storeRepresentation choice under a clockAsk the global-vs-per-key question in the first 90 seconds
Screen B — design60 min, Excalidraw, fault-tolerant job schedulerWhether you find the load-bearing partsAsk at-least-once vs at-most-once. Name the two hard parts at minute 10
Take-home48h, "build something real"Judgement under ambiguityDecision log from hour zero. Walking skeleton by hour 8
Deep diveLine-by-line, from a list written after reading your codeWhether you decided or defaultedDefend every constant. Volunteer the riskiest line
Coding 1Progressive, ~4 gates, pass bar reportedly 2 (assume 3)Representation that survives unseen requirementsTime-to-first-gate ≤ 8 min. Never rewrite
Coding 2Systems-flavored: state, concurrency, memoryJustifying your own choicesInternals arrive as follow-ups to your code
Design — "design ChatGPT"GPU allocation, autoscaling, coordinationScoping judgementAbstract the engine by default; open it in seconds when asked
BehavioralCross-team architecture, consensus under pressureStaff scope, concrete tradeoffsDTAO. Decision in sentence one. A named opponent
Agentic (beta)Multi-file repo, oversized task, drive an agentReviewing a fast confident collaboratorPlan → checkpoints → reject one diff specifically

Calibrate to Staff, not Senior — AI-lab levelling is compressed; the "Senior" title reportedly carries Staff scope. Ask the AI-tool policy per company, per round — policies are opposite at labs you may interview at in the same month.


2. Coding — Representation First

Taught in: tracks/coding/WARMUP.md

2.1 The query-shape table

The single most useful table in the whole document. Hear the query shape, not the noun.

Query shapeStructure
"Is X present?" / "value at exactly K"hash map
"largest key ≤ X"predecessor / floor / as ofsorted array + bisect · balanced tree · skip list
"all keys in [A,B]" — rangesorted array · B-tree · LSM
"smallest element", repeatedlyheap
"least recently used"intrusive doubly-linked list + hash map
"how many in the last N seconds"deque you trim · ring buffer
"seen before, approximately"Bloom / cuckoo filter
"what depends on what"DAG + topological order
"prefix match"trie

A hash map answers exactly, never nearest. The moment a requirement says before, after, as of, range, nearest, at most — you need an ordered structure.

2.2 The three questions

Ask out loud in the first 90 seconds. Each has historically decided whether gate 3 was additive or a rewrite.

  1. "Is the input replayable / immutable?" → if yes, store positions into it, not copies. O(1) checkpoints instead of O(n).
  2. "Is this identifier global or per-entity?" → global makes a snapshot one integer. Per-entity forces a vector and every later gate is harder.
  3. "Will I ever need to undo this?" → if maybe, store deltas, not states. Deltas compose; snapshots do not.

2.3 The ten patterns, one line each

#PatternThe load-bearing idea
1Versioned KV / MVCCAs of V is a predecessor query. Per-key append-only (version, value) list + bisect. Delete is a tombstone. Versions are global. Snapshot = one integer. Compact = reachability from pins
2Delta logStore (cursor_before, n_events, token) per step. Checkpoint = three list lengths. Undo = pop. Redo cleared by a new edit
3Intrusive list (LRU)dict → node; node in a doubly-linked list. Sentinels kill every edge case. Lazy + sampled expiry. Evict until under budget
4Rate limitingFixed window admits at a boundary. Log is exact, O(limit)/key. Counter interpolates, O(1), ~1% error. Bucket separates rate (sustained) from capacity (burst)
5Heap schedulingPush (fire_at, priority, seq, id) — the seq is not optional. Cancel lazily + rebuild at 50% tombstones. Fixed-rate vs fixed-delay + a catch-up policy
6Streaming state machineA carry buffer + a state that survives the chunk boundary. A regex cannot say "no match yet"
7Dependency graphThree colours: WHITE/GREY/BLACK. GREY = back edge = cycle. Two states conflates "on my path" with "done". Reverse graph for incremental recompute
8WAL[len][payload][CRC]. A torn tail is expected, not an error. Checkpoint = temp → fsync → os.replacefsync the directory
9DedupeExactly-once delivery is impossible. At-least-once + idempotent consumer. Bloom errors point the dangerous way for dedupe → use it as a negative cache in front of an exact store
10BackpressureAn unbounded queue is a latency amplifier then an OOM. Four responses: block · buffer · shed · degrade

2.4 Complexity table

StructureLookupInsertDeleteMin/MaxOrdered scan
Hash mapO(1)O(1)O(1)O(n)impossible
Sorted arrayO(log n)O(n)O(n)O(1)O(k)
Balanced BST / skip listO(log n)O(log n)O(log n)O(log n)O(k)
Binary heapO(n)O(log n)O(log n) rootO(1)no
Doubly-linked listO(n)O(1) given nodeO(1) given nodeO(1) endsO(n)
LRU (map + list)O(1)O(1)O(1)O(1) LRUno
TrieO(len)O(len)O(len)prefix O(k)
Bloom filterO(k)O(k)impossibleno
B-treeO(log n)O(log n)O(log n)O(log n)O(k)
LSM treeO(log n)×levelsO(1) amortO(1) tombstoneO(k) merge

Four sentences worth memorizing: hash maps answer exactly, never nearest · heaps keep only the partial order you need · O(1) removal needs the node and double links · append-only data is sorted for free when the key is monotonic.

2.5 The narration script

  1. Restate the problem in your own words.
  2. Clarify — ≥2 questions, ≥1 that could change your representation.
  3. State the approach in two sentences + name the data structure.
  4. State the complexity before implementing.
  5. Write the test for the tricky invariant first.
  6. Code, narrating decisions — not keystrokes. "I'm recording the event count per feed so I can unwind it later" is a decision. "Now a for loop" is not.
  7. When stuck, keep talking. Silence is the costliest narration failure.

2.6 Coding failure modes

SymptomActual gapFix
Time-to-gate-1 > 16 minOver-designing10-minute alarm; gate-1 sprints
Fast G1, rewrite at G3Bad representationBetter clarifying questions, not longer design
Passes tests, can't state complexityNever says it out loudState it before coding, always
Many test runs on the last gateTesting at the endInvariant-first
Narration ≤2Goes silent when stuckNarration-only drill
Cold re-run much slowerMemorized, didn't learnMore variety, less repetition

3. Python Internals

Taught in: tracks/python-internals/WARMUP.md · QUIZBANK.md

3.1 Version-dependent facts

Say the shape, then check — never recite a constant.

FactCurrent state
Free-threadingPEP 703 designed it; PEP 779 defined "supported". Phase II in 3.14 (Oct 2025): officially supported, not default. Phase III unscheduled. sys._is_gil_enabled()
Free-threaded cost~5–10% single-thread overhead (was ~40% at 3.13), ~15–20% more memory, ~4× on suitable multi-threaded CPU work
gc thresholds(gen0, gen1, gen2). Long-documented (700,10,10); (2000,10,10) on 3.13, which also added an incremental collector. Say gc.get_threshold()
CancelledErrorInherits BaseException since 3.8
TaskGroup / except*3.11+
bisect(key=...)3.10+
PEP 479StopIteration escaping a generator → RuntimeError, default since 3.7
PEP 442__del__ in cycles collectable since 3.4
PEP 412Key-sharing dicts — narrows the __slots__ win, so measure
Managed dict3.11+ — instance dict is lazy, so a non-slotted subclass costs nothing until you store in it, then ~5.8×

3.2 Generators

ThingAnswer
Protocol__iter__ + __next__, ends with StopIteration
Iterable vs iteratorIterable returns a new iterator; iterator returns self and holds position
Calling a generator functionReturns a generator object. Runs nothing
What it holdsA suspended frame: locals + instruction pointer + eval stack
send on a fresh generatorTypeError — it's suspended before the first yield; prime with next()
throw outcomes(a) propagates + closes · (b) generator yields → throw returns it · (c) generator returns → StopIteration
close()Throws GeneratorExit at the yield → finally runs. Catching it and yielding again → RuntimeError
yield fromDelegates iteration and forwards send/throw/close and captures the sub-generator's return value
zipOver-consumes — pulls one extra from every iterator before the shortest ends, and discards it
teeBuffers everything one branch read that the other hasn't. Draining one materializes the stream
Memory2M-item list ≈ 77 MiB; generator ≈ 400 bytes. Only cheaper if you never need it twice

3.3 Async

ThingAnswer
Event loopA queue of ready callbacks + one blocking epoll/kqueue call. Single-threaded
Coroutine vs TaskCoroutine is inert. create_task schedules it concurrently. await runs it inline
Fire-and-forgetLoop holds only a weak ref — keep a strong one or use a TaskGroup
gather on failurePropagates the first exception, does NOT cancel siblings → orphans → resource leak
TaskGroupCancels siblings, raises ExceptionGroup, caught with except*. Structured concurrency
CancelledErrorBaseException. except Exception correctly won't catch it. If you catch it, re-raise
CancellationCooperative — delivered at a suspension point. A tight CPU loop can't be cancelled
Blocking callStalls the whole loop. Measured: a 10 ms ticker's max gap goes 10 ms → 162 ms
Escape hatchawait asyncio.to_thread(fn) for blocking I/O · process pool for CPU
Async generatorsCleanup must await, so it can't run in GC. Use contextlib.aclosing or leak connections
asyncio primitivesNot thread-safe. call_soon_threadsafe is the only cross-thread door

3.4 Concurrency decision table

ModelWins onThe cost that makes it lose
asyncioThousands of concurrent I/O waits; high fan-out RPCOne blocking call stalls everything; async all the way down
ThreadsBlocking I/O through non-async libs; moderate concurrencyNo CPU parallelism under the GIL; ~8 MB stack each; shared-state bugs
ProcessesCPU-bound workSerialization per call; memory duplication; slow startup; no shared objects

async for waiting, processes for computing, threads for when the library gives you no choice.

The GIL: guarantees one thread runs bytecode at a time, so single bytecodes and non-releasing C calls are atomic. Does not make your sequences atomic. list.append atomic; counter += 1 not.

The nuance that separates you: since 3.10 the eval breaker is checked only at specific instructions (backward jumps, calls). So a bare counter += 1 loop often loses zero updates — the check lands after the STORE. Put a call between load and store and it loses 3%; make __add__ a Python method and it loses 61%. "I ran it and it didn't lose anything" is not evidence of atomicity.

3.5 Memory

ThingAnswer
Object headerob_refcnt + ob_type. No primitives — an int is a heap object
RefcountingPrompt and deterministic; can't collect cycles; every ref op touches memory
Cycle collectorGenerational mark-and-sweep over containers only. Subtract internal refs; nonzero remainder = live
__del__ in a cycleRuns (PEP 442). Hazards: undefined order, exceptions swallowed, resurrection → prefer weakref.finalize / context managers
AllocatorArenas (256 KB) → pools (4 KB, one size class) → blocks. ≤512 B via pymalloc
"Freed but RSS didn't drop"Expected — an arena releases only when every pool in it is empty
__slots__Removes the per-instance __dict__. Measured ~38% over 200k 3-attr instances
__slots__ breaksNew attributes; weakrefs unless you add '__weakref__'; multiple inheritance from two slotted bases
Subclass trapA subclass omitting __slots__ regains a __dict__. On 3.11+ it's lazy, so identical size until you store in it — then 5.8×. Tell = hasattr(x,'__dict__'), not size
sys.getsizeofOwn footprint only. 50k strings: reports 434 KiB, costs 3,510 KiB. Use tracemalloc
memoryviewZero-copy. 16 MiB slice: bytes copies 16 MiB, view allocates ~0. A live view pins the bytearray
Interning−5..256 cached. 257 folded within one code object. int("257") is int("257") → False. Never use is for values

3.6 Data model

Attribute lookup order: data descriptor on type → instance __dict__ → non-data descriptor → class attrs up the MRO → __getattr__.

ThingAnswer
Data descriptorDefines __set__/__delete__ too → outranks the instance dict → @property can't be shadowed
Non-data descriptorOnly __get__ → instance dict wins → methods can be monkeypatched
Why self bindsA function is a non-data descriptor whose __get__ returns a bound method
__getattribute__Every access. The perf hazard. Classic bug: self.__dict__[name] inside it → RecursionError. Delegate to object.__getattribute__
__getattr__Only on AttributeError fallback. Free on hits. Use for proxies/lazy
super()Not "the parent" — the next class in the MRO of type(self). In Diamond(Left,Right), Left's super reaches Right
MROC3 linearization. Inconsistent → TypeError at class definition
__exit__Returning truthy suppresses the exception. A bare return True eats every bug
__eq__ without __hash__Sets __hash__ = None → unhashable, deliberately

3.7 The traps

TrapWhy
Mutable default argEvaluated once at def, stored on the function object
lru_cache on a methodKeys on self → holds a strong ref to every instance ever → unbounded leak
lru_cache keyingf(1,2) and f(1,b=2) are different keys
String += in a loopImmutable → O(n²). Use "".join
list.pop(0)O(n). Use deque.popleft()
fork with threadsOnly the forking thread survives; a lock held elsewhere is held forever in the child
if k not in d: d[k]=vNot atomic. dict.setdefault is

4. Distributed Systems

Taught in: tracks/systems-design/WARMUP.md

4.1 The arithmetic

Little's law \( L = \lambda W \) — items in system = arrival rate × time in system.

  • Sizing: 50k rps × 8 ms = 400 concurrent.
  • Latency from depth: 10,000 queued ÷ 100/s = 100 s wait.
  • Pool: 500 qps × 20 ms = 10 connections busy.

The utilization knee — M/M/1: \( W = W_s / (1-\rho) \)

ρ0.50.70.80.90.950.99
× service time2.03.35.01020100

Latency is hyperbolic in utilization, not linear. Real traffic is burstier than Poisson, so the knee arrives earlier. This is the whole quantitative argument for admission control.

Retry amplification = \( \sum_{i=0}^{k-1} f^i \)

Failure rate10%50%80%95%100%
3 attempts1.11×1.75×2.44×2.85×3.00×

Storage: 1M × 1 KB = 1 GB · 1B × 1 KB = 1 TB · day ≈ 10⁵ s · month ≈ 2.5×10⁶ s.

4.2 Latency numbers

OperationTime
L1 cache1 ns
Branch mispredict3 ns
L2 cache4 ns
Mutex lock/unlock17 ns
Main memory100 ns
Compress 1 KB (snappy)2 µs
Read 1 MB seq from memory3 µs
SSD random read16–100 µs
Read 1 MB seq from SSD49 µs
Round trip same DC500 µs
Read 1 MB seq from disk825 µs
Disk seek10 ms
RT US cross-country40–70 ms
RT US ↔ Europe80–150 ms

Two derived rules: memory ~100× SSD, SSD ~100× disk seek · any cross-service hop ≥ 0.5 ms, so five sequential hops have a 2.5 ms floor → fan out, don't chain.

4.3 Leases and fencing

The highest-value 200 words in the track.

A lock held by a dead holder is held forever. A lease expires → liveness. But:

t=0   A takes a 30 s lease on J. Starts work.
t=10  A GC-pauses / partitions / is descheduled.
t=30  Lease expires. A hasn't renewed.
t=31  B takes the lease. Runs J.
t=45  B finishes, writes.
t=50  A wakes. From A's view NOTHING HAPPENED. Finishes J and writes.
      A's stale write lands AFTER B's correct one.

You cannot detect this. Unreachable and dead are indistinguishable from outside — that is a theorem. And "check your lease before writing" fails too: the pause can land between the check and the write.

Fencing token — a monotonically increasing number issued with every lease grant. The resource rejects any write with a token below the highest it has seen.

UPDATE results SET value=%s, fence=%s
 WHERE job_id=%s AND fence < %s;   -- 0 rows = superseded. Do not retry.

Where it's checked matters more than the token. The resource must enforce it — not the lock service, not the client. A zombie client believes its token is current. If the resource can't participate (a third-party API with no conditional write), say so and mitigate with idempotency or accept at-most-once.

Lease sizing: renew at lease/3 so two missed heartbeats are tolerable. Too short → GC pauses cause spurious expiry and routine double execution. Too long → a dead holder blocks work. 60 s / 20 s is a reasonable default.

Redlock: relies on bounded clock drift and bounded pauses for correctness; neither is guaranteed. The safety argument lives in fencing, not the lock protocol.

4.4 Replication and quorums

ModeAck whenOn failoverLatency
Syncall replicaszero lossslowest replica
Asyncleader onlyacked writes can be lostfastest
Semi-sync≥k replicaslose only if >k fail togetherone slow replica tolerated

Quorum: \( W + R > N \) forces overlap (pigeonhole) → a read sees the latest write.

NWRProperty
322Standard, tolerates 1 failure both ways
331Fast reads, no write availability on any failure
533Tolerates 2 failures
311W+R=2 ≤ 3 → no overlap, eventual only

What a quorum does NOT give you (name three): sloppy quorums break the overlap · concurrent writes still need conflict resolution · a write that fails after reaching some replicas isn't rolled back · read-your-writes isn't guaranteed across sessions · quorum reads aren't linearizable without read-repair-then-commit.

Convergence: read repair (cheap, on the read path, misses cold data) + anti-entropy (Merkle trees — compare root hashes, descend only into differences, O(log n) in the difference).

4.5 Consensus

What it buys: agreement on an ordered log → replicated state machine → anything.

FLP: in a fully asynchronous system with one faulty process, no deterministic algorithm guarantees consensus. Practical systems add timeouts → safety always, liveness under assumptions.

Raft:

  • Terms are a logical clock. Higher term seen → step down. That one rule kills most split-brain.
  • Randomized election timeouts (150–300 ms) break split votes without coordination.
  • An entry is committed once on a majority.
  • Log matching: same index + same term ⇒ identical logs up to that point (by induction).
  • Safety rule 1 — election restriction: a voter refuses a candidate whose log is less up-to-date. Two majorities must intersect ⇒ any winner has every committed entry.
  • Safety rule 2 — never commit a previous term's entry by counting replicas. A leader commits only its own term's entries by counting; older ones commit indirectly. In practice a leader appends a no-op to trigger it. (This is the part people skip.)

Costs — say these: every write is ≥1 RT to a majority (~1 ms same-DC, 50–150 ms cross-region) · the leader is a throughput ceiling → shard into many Raft groups · failover is a latency spike · odd numbers only (3→4 doesn't improve tolerance) · membership change is the hard part.

Use consensus for metadata, not data.

4.6 Consistency models

LinearizabilitySerializability
Aboutsingle objectsmulti-object transactions
Guaranteesrecency (real-time order)isolation (equivalent to some serial order)
Silent abouttransactionsreal time

Strict serializability = both. That's Spanner, and why it needs TrueTime.

Snapshot isolation permits write skew — two txns read overlapping data, write disjoint keys, both commit, jointly break an invariant. Canonical: on-call doctors. Fixes: SELECT ... FOR UPDATE · materialize the conflict · SSI (Postgres SERIALIZABLE).

Session guarantees (cheap, and what users notice): read-your-writes · monotonic reads · consistent prefix · causal · bounded staleness.

CAP, precisely: when a partition occurs, choose consistency or availability. Only during a partition. "Available" means every non-failing node responds. Say PACELC: if P then A or C, Else Latency or Consistency.

4.7 Time

ClockUse forNever
Wall (time.time)timestampsmeasuring durations — NTP steps it
Monotonic (time.monotonic)durations, leasescomparing across machines

NTP holds a few ms on a good LAN — a statistical claim, not a bound. A node cannot know its own skew.

MechanismGivesCannot
Lamporttotal order consistent with causalitydetect concurrency
Vector clocksdetects concurrency (conflicts)O(nodes) size; pruning is subtle
HLCclose to physical + respects causality + O(1)detect concurrency
TrueTimebounded uncertainty intervalbe cheap — commit-wait pays it in latency

Last-write-wins by wall clock is a documented data-loss mode when a node's clock is skewed.

4.8 Partitioning

HashRange
Distributionevenuneven, hot spots easy
Range queriesimpossibleefficient
Resizeneeds consistent hashingsplit ranges

hash(key) % N → changing N moves ~80% of keys. Consistent hashing → only K/N move, and only from one neighbour. Virtual nodes (100–256/node) fix two things: load variance (drops as 1/√v) and — more important — a node's failure spreads across many successors instead of dumping its whole range on one and cascading.

Hot partitions: cache it (usually the whole answer) → split the key (id:0..99) → dedicated partition → rate-limit it. Per-key metrics come first — you can't fix what you can't see, and aggregates hide it completely.

Rebalancing: snapshot → stream delta → briefly block + flip ownership. Fence the flip. Never auto-rebalance on failure (a blip triggers a storm). Rate-limit the copy.

4.9 Delivery semantics

Exactly-once delivery is impossible — sender can't distinguish "never arrived" from "arrived, ack lost"; more round trips just move it to the ack of the ack (Two Generals).

Exactly-once processing = at-least-once + idempotent consumer. Requires a stable idempotency key generated by the PRODUCER, unchanged across retries. Generate it at send time and dedupe silently does nothing.

Dual writedb.save(x); queue.publish(x) — has no safe ordering. Fix: outbox — insert into an outbox table in the same transaction, a relay publishes with FOR UPDATE SKIP LOCKED. Or CDC (tail the WAL/binlog).

DLQ needs four things, and most designs mention only the first: the reason (error, stack, attempt count) · a replay path · poison detection (permanent vs transient — don't retry 4xx) · alert on arrival rate, not depth. And state the ordering consequence: if message 5 dead-letters, 6 either blocks (order preserved) or proceeds (order broken). No third option.

4.10 Load control

Fix order — most people get this backwards:

  1. Retry budget — cap retries at ~10% of base traffic. Bounds amplification at 1.1× no matter how bad it gets. (gRPC retry throttling, Envoy retry budgets.)
  2. Circuit breaker — stop trying entirely.
  3. Jitter — desynchronize what you do send.

Jitter alone still delivers 2.85× at a 95% failure rate. And don't retry at every layer — 3 layers × 3 retries = 27 attempts.

Backoff variants: full uniform(0, min(cap, base·2^n)) — AWS's simulation found it minimized both total work and completion time · equal temp/2 + uniform(0, temp/2) — keeps a floor · decorrelated uniform(base, prev·3) — smoothest, hardest to bound.

Circuit breaker: closed → open (fail immediately) → half-open (one probe, not a flood). Threshold must be a rate over a minimum volume ("50% over ≥20 requests in 10 s") — an absolute count breaks on low-traffic endpoints. Scope per endpoint, often per instance.

Shedding: rejecting 10% in 1 ms beats accepting 100% and timing all out at 30 s. Shed by priority, not at random. Shed the OLDEST queued item — under sustained overload FIFO serves only requests whose clients have given up. Deadline propagation: pass the remaining budget downstream; fail fast if there isn't enough left.

Bulkheads: separate pools per dependency. Honest counter-argument: by M/M/c, one pool of 50 has better tails than five of 10 — you trade efficiency for isolation. Cellular architecture is the strongest blast-radius answer: 1/N by construction.

4.11 The failure catalog

FailureDetectionContainmentRecovery
Node crashheartbeat / lease expirytraffic drainsreplacement joins; re-replicate
Fail-slowlatency percentiles vs peers, outlier detectioneject on latency SLO, timeouts everywhererestart. Never trust its self-report
Partitionquorum loss on the minorityminority refuses writesmerge + reconcile on heal
Zombie holderundetectablefencing token rejected at storagenothing to recover
Thundering herdqueue depth spikerate-limited catch-up, jittered restartsbounded drain
Retry stormrate up while success downretry budgetcircuit break → half-open probe
Poison messageattempt countDLQ after Nreplay after fix
Hot partitionper-key metricssplit / cache / limitrebalance
Cascading failurecorrelated latencybulkheads, timeouts, sheddingshed → ramp
Corruptionchecksums, invariant auditsquarantine, stop replicatingrestore to known-good
Clock skewskew monitoringtreat as unhealthyresync; re-elect
Bad config rolloutcanary divergencestaged rolloutauto-rollback

Fail-slow is worse than fail-stop and it's the common case — the node answers your health check in 2 ms while serving real requests in 40 s. If your only health signal is "does it respond", you haven't handled the common case.

Say one deliberately accepted failure mode with its cost. That's a staff move; claiming to have handled everything is falsifiable in one question.

4.12 The design template and clock

1 Requirements & scope   (incl. explicitly out of scope)
2 Scale numbers          (and the arithmetic you did)
3 API surface            (3-5 calls that matter)
4 Data model             (keys and indexes, and WHY)
5 High-level arch        (the diagram)
6 DEEP DIVE: the two hardest components
7 Failure & recovery     (detection · containment · recovery, each)
8 Bottlenecks & evolution
9 Tradeoffs explicitly rejected
MinDo
0–5Clarify. Scale numbers written down
5–10API + data model
10–20Architecture + diagram
20–35Deep dive on the two hard parts
35–45Failure, bottlenecks, rejections

Finding the two hard parts: where must ≥2 machines agree? · where can data be lost? · what's highest-rate/cardinality? · where does one tenant affect another?

Say at minute 10: "I think the two places this can actually fail are X and Y, so that's where I want to spend the time — does that match what you care about?"


5. Inference Infrastructure

Taught in: tracks/ml-infra/WARMUP.md · runnable: gpu_math.py

5.1 The one derivation

Arithmetic intensity \( I = \text{FLOPs} / \text{bytes moved} \). Machine balance = peak dense FLOP/s ÷ bandwidth. H100: \( 989.5 \times 10^{12} / 3.35 \times 10^{12} \approx \mathbf{295}\) FLOP per byte.

Decode: every weight is read once per step regardless of batch; each sequence does ~2N FLOPs.

\[ I_{\text{decode}} \approx \frac{2Nb}{2N} = b \qquad\text{(the batch size)} \]

At batch 1 you use ~1/295 of the GPU's compute. Say "dense". The datasheet's 1,979 TFLOP/s is with 2:4 sparsity; LLM weights are dense, so the real number is 989.5 and the balance is 295, not 590. Quoting the sparse figure for a dense workload is a cheap way to look like you read a spec sheet instead of a benchmark.

70B FP16 on H100:
  weights 140 GB / 3.35 TB/s   = 41.8 ms   (memory)
  2 x 70e9 / 989.5e12          =  0.14 ms  (compute, dense BF16)
  ratio 295x  → MEMORY-BOUND   (exactly the machine balance — a good self-check)

Prefill: all s prompt tokens at once → \( I \approx s \). A 2,000-token prompt = 280 TFLOP = 141 ms of solid computecompute-bound, and it blocks everyone's decode.

Prefill is compute-bound, decode is memory-bound. Two workloads on one accelerator. That sentence is most of this round.

The proof: H200 has identical compute to H100 (989.5 TFLOP/s BF16 dense) and +43% bandwidth (4.8 vs 3.35 TB/s), and is materially faster at decode. If decode were compute-bound it would be exactly as fast.

5.2 Memory formulas

QuantityFormula
WeightsN × bytes/param
KV per token2 × layers × kv_heads × head_dim × bytes
Max batch(HBM − weights − activations) / (kv_per_token × seq_len)
Decode step floor(weight_bytes + kv_bytes) / bandwidth
Decode intensity≈ batch
Prefill intensity≈ prompt_len
Prefill FLOPs≈ 2 × N × prompt_tokens
Decode FLOPs/token≈ 2 × N
PrecisionB/param70B
FP324280 GB
FP16/BF162140 GB
FP8/INT8170 GB
INT40.535 GB

5.3 Hardware numbers

GPUMemoryBandwidthBF16FP8
A100 80GB80 GB HBM2e2.04 TB/s312
H100 SXM80 GB HBM33.35 TB/s989.51,979
H200 SXM141 GB HBM3e4.8 TB/s989.5 — same1,979
B200192 GB HBM3e~8 TB/s~4,500~9,000 (FP4)

Cloud $/hr, 2026-reported, order of magnitude: H100 ~$1.50–3.00 · H200 ~$3.80 · B200 ~$6.50. Always attach a date to a price.

5.4 Llama-70B anchors

Weights FP16140 GB → needs ≥2 H100s, 4 in practice
KV/token (L=80, kv_heads=8, d_h=128)320 KB
KV @ 4k context1.25 GB/sequence
KV @ 128k context39 GB — half an H100 for one user
GQA saving vs MHA (2.5 MB/token → 320 KB)
4×H100 available for KV320 − 140 − 16 = 164 GB
Max batch @4k~130
Decode step @ that batch304 GB read → 91 ms
Aggregate~1,440 tok/s
Per user~11 tok/s ≈ reading speed (sanity check!)
Cost/1M output tokens~$5 (4×H100 @ $2.50/hr, 40% realized)

At batch 130 the KV cache (164 GB) exceeds the weights (140 GB). Doubling context halves concurrency, one for one.

5.5 Technique tradeoffs

TechniqueBuysCostsLoses when
Continuous batching (Orca)no idle slots, no HOL blockingscheduler complexity, TPOT jitter~never — table stakes
PagedAttention (vLLM)near-zero KV fragmentation → bigger batch; CoW prefix sharingindirection per attention op, custom kernel~never
Chunked prefill (Sarathi)much better TTFT/TPOT tailsslightly lower prefill throughputthroughput > tails
Prefix cachingskips prefill for shared prefixes; O(n²)→O(n) in a chatcache memory competes with KV; eviction policy; must be tenant-scoped or it leaksprefixes aren't shared
Speculative decoding1.5–3× latency at low batch; output distribution provably identicalwasted compute on rejects; a second modelhigh batch — no spare compute
Quantizationhalves the dominant bytes → helps decode twicequality, workload-specificquality is the product
Disaggregated P/Deach scales independentlyKV transfer over the networktransfer > interference avoided

Parallelism: TP splits each layer — all-reduce every layer, needs NVLink, within a node, improves latency. PP splits layers — one transfer per boundary, across nodes, bubbles, does not improve latency. EP for MoE — all-to-all per layer, expert load imbalance is the hot-partition problem again.

The framing that scores: these aren't a stack of free wins, they're points on a throughput-vs-tail-latency curve — and it isn't one curve. Interactive chat, agentic loops, and batch want different scheduler policies → separate pools or one priority-aware scheduler with preemption. (Recompute is usually cheaper than swapping KV over PCIe.)

5.6 Autoscaling signals

Request cost varies ~10,000× ("hi" ≈ 1 unit; a 100k-token agent step ≈ 50,000).

SignalVerdict
Requests/sec✗ nearly uncorrelated with load
GPU utilization✗ misleading — high while memory-stalled
Queue depth / wait✓ direct measure of unmet demand
Tokens/s (prefill and decode separately)✓ the real unit of work
KV cache occupancybest leading indicator — the binding constraint, rises before queueing
TTFT/TPOT p95✓ good SLO trigger

Composite: max(kv_occupancy/0.85, queue_wait_p95/target).

Scale-up is minutes (instance acquisition + pulling + loading 140 GB of weights + CUDA graph capture) → reactive is always late. Forecast from the diurnal curve · warm pools sized by forecast error, not average load · admission control as the fast path.

Fairness on TOKENS, not requests — ten 100k-token requests is 1,000× ten small ones.

5.7 The two altitudes

Default (abstracted) — say this verbatim:

"I'll treat the inference engine as a service with three properties: it exposes capacity in tokens per second rather than requests per second, it has an admission interface I can apply backpressure to, and it streams. I'll spend my time on traffic, coordination and failure — tell me if you want me to open it up."

When they say "open it up", lead with the constraint, not a build-up:

"The binding constraint is memory bandwidth, and the binding capacity is the KV cache."

Then: memory budget → why decode is bandwidth-bound (+ H100/H200 proof) → therefore batching → continuous batching + PagedAttention → the prefill/decode conflict → chunked prefill → prefix caching → the curve framing.

Never claim knowledge of their internal stack. Say: "I don't know how you do this internally, so I'll reason from the public systems — PagedAttention, Orca's continuous batching, Sarathi's chunked prefill — and you can tell me where that diverges."


6. Take-Home and Deep Dive

Taught in: tracks/take-home/WARMUP.md

The take-home and the deep dive are ONE round. The take-home generates a personalized interrogation surface. Every decision in the 48 hours is a question in week three. So the target isn't "best code" — it's code every line of which I can defend, plus a written record of the alternatives I rejected.

HoursPhase
0–2Interrogate the brief → written list of ambiguities + decisions
2–4Design doc v1
4–8Walking skeleton, committed, green
8–28Implementation with tests as you go
28–34Sleep. Non-negotiable
34–40The hard part you deferred
40–44One benchmark + methodology
44–47README, design doc v2, commit history
47–48Buffer

Never cut: tests + a one-command runner · README that works on a clean machine · design doc with tradeoffs · ambiguities section · decisions.md · error handling on external boundaries · clean commit history · one benchmark · "what I'd do with two more days".

"Beyond the ask" = ONE of: a measured benchmark with honest methodology (including the disappointing number) · a failure-injection test that proves a recovery path · an operational concern nobody asked for (structured logs + correlation ID, real health check, DLQ runbook). Not more features — extra features read as poor judgement.

Cut order: extra features → configuration breadth → non-benchmark optimization → admin/UI → persistence sophistication. Write down what you cut and why — a documented cut is a decision.

Decision log entry — five fields:

Decision · Alternatives (with numeric reasons) · Assumes · Would revisit if · Not tested

The last two are what make it interview-grade. Write one for every constant.

The seven interrogation classes: Choice ("why X not Y") · Magic number ("why 30 s") · Scale ("at 100×") · Data loss ("where can this lose a message") · Omission ("what didn't you test") · Regret ("what would you change") · Hostile ("this function does four things").

The closing move — unprompted: name the riskiest line in your own diff and the next thing you'd build. Volunteering your design's weakest point before it's found is the single most credibility-generating move in the round.


7. Behavioral

Taught in: tracks/behavioral/WARMUP.md

DTAO, not STAR — STAR buries the decision.

SectionShare
DDecision, one sentence, first1 sentence
TTradeoff — alternatives + why each lost. Numbers~40%
AAlignment — who disagreed, what you did~30%
OOutcome — measured, including what you got wrong~25%

Context goes in a clause: "On the multilingual ranking pipeline, we decided X."

Four sentences that make it Staff:

  1. "The decision was ___." (first)
  2. "I rejected ___ because ___." (with a number)
  3. "___ disagreed, and their argument was ___." (stated fairly)
  4. "I got ___ wrong." (specific, generalizable lesson)

The test: if nobody disagreed, it isn't a Staff story.

Twelve required categories: cross-team architecture · a disagreement you lost · one you won · an outage you owned · a project you killed · raising a team's bar · a bet that failed · consensus without authority · deadline vs quality · changed your mind from data · non-engineers · what you'd build differently.

The disagreement-you-lost failure modes: humble-brag ("six months later they did it my way") · victim ("political reasons") · trivial · revisionist ("they were right about everything").

The seven probes — and the first is the discriminating one:

  1. "What was their strongest argument?" ← if you can't produce one, the story is suspect
  2. "What would have made the other option win?"
  3. "Who else was affected that you didn't mention?"
  4. "What did that cost the other team?"
  5. "How much of the timeline was the disagreement?"
  6. "How do you know it wasn't a coincidence?"
  7. "What would you do differently?" ← never answer "communicate more"

Never: lead with situation · a story with no disagreement · strawman the opposition · say "we" throughout · present single-team as cross-team · perform humility or enthusiasm · over-rehearse (recited is audible).


8. Agentic Coding

Taught in: tracks/agentic/WARMUP.md

The eight steps: baseline first (pytest, record count + runtime) → read ≤10 min → write the plan (end state, 3–5 verifiable checkpoints, the invariant, what you'll do manually) → delegate in checkpoint units → verify every checkpoint, read the diff even when greenreject specifically → take over after two failed attempts on the same checkpoint → narrate continuously.

Prompt properties: scope ("do not modify other files") · contract (exact signatures + validation) · invariant ("all N existing tests pass; behaviour with X=None byte-identical") · verification ("add tests/test_x.py covering…") · domain knowledge it lacks ("close the response body before retrying or we leak connections") · negative space ("do not catch CancelledError — it's a BaseException").

Never re-prompt with "that didn't work." Say what failed, why it's wrong at a domain level, and what to do instead.

Take over when: two failures on one checkpoint · the invariant isn't expressible as a test · you need to understand it to defend it later · a plausible-looking wrong answer is worse than slow progress. Don't take over the 30-file rename — that's the inverse error.

A smaller verified diff beats a larger unverified one. One specific rejection is worth more than three accepted diffs.


9. The Scripts

Verbatim sentences worth having ready.

Coding — clarifying (pick the one that changes your representation):

"Are versions global or per key?" · "Is the source replayable and deterministic?" · "Will I need to undo this?" · "Do you want events returned per call or accumulated?"

Design — scoping, at minute 10:

"I think the two places this can actually fail are X and Y, so that's where I want to spend the time — does that match what you care about?"

Design — at-least-once, obligated follow-through:

"That selects at-least-once — which means job handlers must be idempotent, and I'll give each execution a stable idempotency key so they can be. Exactly-once execution of a side-effecting job isn't achievable without cooperation from the job itself, so I won't claim it."

Design — the zombie:

"I can't distinguish a dead worker from an unreachable one — that's a theorem, not a gap in my design. So I'm choosing at-least-once and making the duplicate safe with a fencing token checked at the storage layer."

Design — accepting a failure mode:

"If we lose a region mid-write, in-flight requests are lost. I'm accepting that: synchronous cross-region replication adds 80 ms to every write, which blows the 200 ms p99 for the 99.99% of the time there's no regional failure. I replicate async and expose an RPO of ~5 seconds. If the business needs RPO zero, that's a different design and a different latency budget."

Inference — the abstraction:

"I'll treat the inference engine as a service with three properties: capacity in tokens per second, an admission interface I can apply backpressure to, and streaming. I'll spend my time on traffic, coordination and failure — tell me if you want me to open it up."

Inference — the boundary of your knowledge:

"I don't know how you do this internally, so I'll reason from the public systems — PagedAttention, Orca's continuous batching, Sarathi's chunked prefill — and you can tell me where that diverges."

Deep dive — the closing move:

"Two things you haven't asked about. The riskiest thing in here is ___ — it's correct but it's where a future change silently breaks ___, and it's the least-covered path. And the thing I'd build next isn't a feature, it's ___."

Behavioral — a number you can't defend:

"I'll be honest that it's a judgement call rather than a measurement — if I had production data I'd set it from the observed distribution."

Any round — being wrong:

"You're right. (pause) The reason I did it that way was ___, which is a weak reason. What I'd do instead is ___."

Any round — a performance claim:

Either "I measured 2.8× on my harness at batch 32 with a 512-token prompt" or "vLLM's published benchmarks report 3–5×". Never assert an unattributed number.


10. Pre-Round Checklist

The week before

  • ☐ Ask the recruiter the AI-tool policy, per round — policies are opposite at different labs
  • ☐ Confirm the tooling (CoderPad? Excalidraw? their environment?) and practise in it
  • ☐ Read the charter/core-views in a browser; fill the [VERIFY] slots
  • ☐ Read their two most recent engineering posts; write one specific question about each
  • ☐ Two consecutive full-loop sims at hire (staff) or better
  • review/ queue clear, no leeches
  • ☐ All six forward-looking answers rehearsed within 7 days

The morning of

  • ☐ Read only this file's section for that round
  • ☐ Timer visible · recorder on for practice, off for the real thing
  • ☐ Water. Eat. The screen is two 60-minute rounds back to back and fatigue is measurable

In the first two minutes of any technical round

  • ☐ Restate the problem
  • ☐ Ask the question that could change your approach
  • ☐ State scale assumptions out loud, with numbers
  • ☐ Name what's out of scope

In the last two minutes

  • ☐ State what you'd do next and why you didn't
  • ☐ Name the weakest part of what you built
  • ☐ Ask one of your three prepared questions

Every claim here is taught from first principles in the WARMUP guides, and every number is verified — see STATE.md for what was measured and when.