The Concept Map — What Recurs Across All Fifteen

Fifteen projects, but far fewer mechanisms. The same handful of ideas appear at different altitudes, wearing different names, and the point of building all fifteen is that you stop seeing fifteen problems and start seeing one problem with fifteen surfaces.

This page is the index of that claim. Every row is a mechanism that appears in three or more projects, with the specific place it appears and the name it goes by there.

Contents

• The eight recurring mechanisms
• One: an approximate test guarding an exact one
• Two: atomic pointer swap as the durability primitive
• Three: the memory hierarchy is the cost model
• Four: maxima, tails and fan-out
• Five: two-stage retrieve-then-rank
• Six: restriction as the enabler of recovery
• Seven: the accuracy/latency dial
• Eight: amortising a fixed cost
• The numbers every project leans on
• Where each project's real difficulty lies
• What the seven refuted predictions have in common

The eight recurring mechanisms

One: an approximate test guarding an exact one

The pattern: an exact operation is expensive because it touches slow storage. Put a cheap, approximate, one-sided test in fast memory in front of it.

The design question is always the same: what is the error direction, and is it recoverable? A Bloom filter's error costs one wasted read. A candidate generator's error costs the result entirely. That asymmetry decides how much budget each deserves — and it is why P08 block 7's ceiling is the first thing to check when a recommender misses.

Two: atomic pointer swap as the durability primitive

Every durable system in this track reduces to the same three lines: write the new state somewhere else, make it durable, then flip one pointer atomically.

write payload to a temporary location
fsync                                  (~90-105 us, measured)
rename / CAS the pointer               (atomic by construction)

Once you see it, "exactly-once" stops being mysterious. P06 runs tasks at least once and renames atomically, so the effect is once. P07 processes records twice after a crash and advances state-and-offset together, so the effect is once. The guarantee is never about delivery; it is about the pointer.

Three: the memory hierarchy is the cost model

Measured on this machine (numbers.md), the ratios that decide every design in the track:

Eight orders of magnitude, and the boundary you cross determines the architecture:

• Cross DRAM → the LSM exists (P04), tiling exists (P14), quantisation exists.
• Cross fsync → batching and group commit are mandatory (P04, P03, P05).
• Cross HDD seek → graph search becomes impossible and IVF wins (P02).
• Cross the network → consensus costs a round trip and geography sets the floor (P05).

The corollary that catches everyone: the same code changes regime when the data grows. P14's assembly measures a 25 MB working set behaving as if it were free, and the same code at 100 MB paying full DRAM cost — with the roofline ratio crossing 1.0 in between.

Four: maxima, tails and fan-out

One piece of arithmetic, four appearances. If an event has independent probability \(p\) per trial and there are \(n\) trials, \(P(\text{at least one}) = 1 - (1-p)^n\).

The mitigations rhyme too: redundancy plus cancellation. Backup tasks (P06), hedged requests (P15), larger quorums that can exclude a straggler (P05). And the statistical version — Bonferroni, Benjamini–Hochberg — is the same acknowledgement that \(n\) trials need a stricter per-trial bar.

Five: two-stage retrieve-then-rank

Cheap and wide, then expensive and narrow. Stage 2 can never exceed stage 1's ceiling, which is the single most useful diagnostic in any such system.

P08 block 7 makes the failure explicit: recall after re-ranking tracks the stage-1 ceiling exactly, and no ranker improvement can cross it. Before blaming a ranker, check whether the item was in the candidate set.

Six: restriction as the enabler of recovery

Each of these systems gives up expressiveness and gets fault tolerance in return. The restriction is the feature.

P06's assembly is the cleanest demonstration: byte-identical output under three different worker-kill schedules. Allow one impure map function and retry, speculation and rescheduling all become unsound simultaneously.

Seven: the accuracy/latency dial

In four projects, correctness is a parameter rather than a property, and the engineering task is to make the parameter explicit rather than accidental.

The mature version of this is not picking a value; it is shipping the dial and labelling each setting with its measured error, as P15 block 6 does when it reports the dashboard's bias alongside its latency.

Eight: amortising a fixed cost

When an operation has a large fixed cost and a small marginal one, batching is not an optimisation — it is the design.

Note the last two: the "batch" is not always requests. Huge pages batch translations; memory-level parallelism batches outstanding misses. The pattern is the same — pay the fixed cost once for many units of work.

The numbers every project leans on

If you memorise one table from this track, make it this one — every design decision above is a comparison between two of its rows.

Where each project's real difficulty lies

Not where the tutorials put it.

What the seven refuted predictions have in common

Every one of them (listed in the index) was a case of importing a conclusion without importing the conditions that made it true:

• P08's pop^0.75 — a constant that is correct in word2vec, where discounting frequency is the goal, copied into a domain where popularity is the signal.
• P09's Thompson sampling — a result proven for single-action bandits, applied to a slate where 34% of attention sits in one slot.
• P11's bytecode VM — a design whose benefit comes from a jump table, hosted in a language with no jump table.
• P12's working-set cliff — a model of a single phase, applied to a mixture of phases.
• P14's single roofline — a machine constant that is actually a per-dtype constant.
• P13's broadcast failure mode — an assumption about numpy's behaviour that was simply never run.
• P08's negative count — folklore repeated without a control.

The generalisable habit: when you import a result, import its conditions and check that they hold. Every one of these was caught by a measurement that took minutes, and none would have been caught by reading more carefully.

Start with the hands-on index, or go straight to a project's page from the table above.