The Cross-Cutting Map
Six mechanisms account for most of what the five hands-on pages measure, and each shows up on three or more of them wearing different clothes. Recognising the same mechanism across substrates is the transferable skill; the pages are only where it was measured.
Table of Contents
- The Six Recurring Mechanisms
- 1. The Ack Position, and Why There Is No Third
- 2. Time-of-Check to Time-of-Use
- 3. Fencing and Epochs
- 4. The Saturating Signal
- 5. Reserved Floors Over Strict Priority
- 6. Approximate in Fast Memory, Exact in Slow
- The Numbers Worth Memorising
- What Every Page Ends Up Saying
The Six Recurring Mechanisms
| Mechanism | C01 | C03 | C05 | C11 | M02 |
|---|---|---|---|---|---|
| The ack position | B1–B2 | — | B5 (deadline) | B2 (lease expiry) | — |
| TOCTOU | B5 (dual write) | B6 (GET-then-SET) | — | B4 | — |
| Fencing / epochs | B6 | — | — | B3 | — |
| The saturating signal | — | — | B3 | — | B6 |
| Reserved floors | — | B5 (per-key) | B6 | — | — |
| Approximate then exact | B4 (Bloom) | B4 (counter) | — | — | B5 (tiering) |
1. The Ack Position, and Why There Is No Third
The shape: an effect and a record-that-the-effect-happened are two events. One must come first. A failure between them is the only failure, and which side you lose on is decided entirely by the ordering.
| Page | The two events | Before → | After → |
|---|---|---|---|
| C01 B1–B2 | apply the job / ack the queue | 1.99% lost | 1.99% duplicated |
| C01 B5 | apply the effect / write the dedup key | — | duplicates escape at exactly the crash rate |
| C11 B2 | acquire the lease / do the work | lease expires mid-work | split brain |
Why it matters: candidates try to find a third position. There is not one — this is the two-generals result, and recognising it saves the ten minutes otherwise spent inventing a protocol that cannot exist.
What to do instead: pick at-least-once and make the second application a no-op. C01 B3 measures that it works; B5 measures the one condition it requires — the dedup key must be written in the same transaction as the effect, or the window has only moved.
2. Time-of-Check to Time-of-Use
The shape: a check establishes a fact about the past; the action depends on a fact about the present. Any delay between them is a window, and the window's size is a latency you usually do not control.
| Page | The check | The gap | Measured |
|---|---|---|---|
| C03 B6 | GET the counter | network RTT before the SET | 10 admitted against a limit of 5 |
| C11 B4 | read the current fence token | any pause before the write | correct or wrong depending only on where the pause lands |
| C11 assembly | as above, swept | 1 ms → 1 s | leaks 1.2% → 72.5% |
| C01 B5 | "have I seen this job?" | the crash window | dedup catches 0 of the crash duplicates |
The rule: a check and the effect it guards must be atomic, which means they must happen at the same component. Any design where X validates and Y acts has this window.
How to find it in a design: ask which component orders the operations.
That is the one the check belongs in. In C03 it is Redis (INCR, not
GET-then-SET); in C11 it is the resource (fence check inside the write); in
C01 it is the database (dedup key in the same transaction).
3. Fencing and Epochs
The shape: any identity that can be reused, superseded or restarted must carry a monotonically increasing number, and the component that acts on the identity must reject anything below the highest it has seen.
| Page / design | The identity | The epoch |
|---|---|---|
| C11 B3 | lock holder | fence token from acquire |
| C01 B6 | job lease | visibility-timeout generation |
| d02 | shard owner | rebalance epoch |
| m03 R5 | a node hostname | boot epoch |
| Raft | leader | term |
| ZooKeeper | session | zxid |
The measured claim (C11 B3): with fencing, the stale writer's update is rejected and the correct value survives — and it is rejected without the resource talking to the lock service, which is what makes it robust to the lock service being slow, partitioned or down.
The honest limit (C11 B3, beyond the toy): fencing requires the resource to cooperate. A third-party API will not check your token. When it cannot, you do not have a safe design — you have a probabilistic one, and the correct move is to say so and reach for idempotency instead.
4. The Saturating Signal
The shape: the metric everyone reaches for is bounded above, and it saturates exactly at the point where you need it to keep moving.
| Page | The tempting signal | Where it dies | What to use |
|---|---|---|---|
| C05 B3 | CPU utilisation | pinned at 100% from 99 rps to 120 rps, while queue depth goes 51 → 1,617 | queue depth, or measured wait |
| M02 B6 | GPU utilisation | reads ~100% across the whole preemption cliff | KV occupancy |
| C03 B1 | request count | 1 request/min at 128k context vs 1000/min at 200 tokens | tokens, or KV·seconds |
The general form: the utilisation of a resource is not the scarcity of that resource. A signal fit for shedding must be unbounded above and must lead rather than trail. Error rate fails the second test; utilisation fails the first.
5. Reserved Floors Over Strict Priority
The shape: two classes contend; strict priority protects the top class by starving the bottom one to zero, and zero is not a degradation, it is an outage for that class.
| Page / design | Classes | Strict priority gives the bottom class |
|---|---|---|
| C05 B6 | premium / free | 5.8% completion — measured |
| d05 | shed classes | starvation under sustained load |
| m01 | enterprise / standard / free | and its revision: a floor must guarantee latency, not merely admission |
| m07 | hot / long-tail adapters | the tail never enters a batch |
| m03 | quota tiers | large jobs never schedule without aging |
C05 B6 measures the middle ground: a floor recovers 87% of strict priority's gain for the top class while taking the bottom class from 5.8% to 39.2%.
The question to ask whenever a design reaches for priority: what is the bottom class guaranteed? If the answer is "nothing", it will eventually get nothing — and the guarantee you can write in a contract is the floor, not the priority.
6. Approximate in Fast Memory, Exact in Slow
The shape: an exact answer is expensive; a cheap approximate test that is wrong in only one direction guards it.
| Page | The approximation | The direction it is safe in |
|---|---|---|
| C03 B4 | two counters instead of a timestamp log | over-estimates recent load → over-admits, bounded |
| C03 B5 | local lease, then the store | over-admits by lease × processes |
| C01 B4 | Bloom filter before the dedup table | must answer definitely new; a false positive falls through to the exact check |
| M02 B5 | fetch a cached prefix, else recompute | a miss costs recompute, never wrong output |
The direction is the whole design. A Bloom filter used the wrong way round — treating "probably seen" as "seen" — skips a job, which is data loss. The same structure is safe or unsafe depending only on which way the error points, and C01 B4's beyond the toy is the worked version.
The Numbers Worth Memorising
These come up in more than one round, and every one was measured on its page.
| Number | What it is | Page |
|---|---|---|
| 2× | what a fixed window admits at the boundary — and the sliding counter's worst case | C03 B1, B4 |
| 8.7× | p99 latency increase from 50% to 95% utilisation | C05 B1 |
| 1/(1−ρ) | queueing delay as a multiple of service time | C05 B1 |
| 62× | goodput gain from dropping work whose deadline has passed | C05 B5 |
| 320 KiB | KV cache per token, 70B with GQA-8 at fp16 | M02 B1 |
| 23% | share of a 4×H100 replica held by one 128k-context request | M02 B1 |
| 9.3 GB/s | break-even to fetch cached KV rather than recompute, TP4 | M02 B5 |
| 6× | batch increase from paged over contiguous KV allocation | M02 B3 |
| 989.5 TFLOP/s | H100 BF16 dense — the datasheet's 1,979 is with 2:4 sparsity | M02 B5 |
| ~0 | cost of a fence check at the resource | C11 B3 |
What Every Page Ends Up Saying
Read together, the five reach the same conclusion from five directions:
Put the check where the ordering happens, and make the failure cheap rather than rare.
- C01: the dedup key goes in the transaction that performs the effect — not in the worker, not in the queue.
- C03: the decrement goes in the store that orders the requests — one
INCR, notGET-then-SET. - C05: the deadline check goes at dequeue, where the capacity is about to be spent — not at enqueue, where nothing has waited yet.
- C11: the fence check goes in the resource that orders the writes — not in the client that is about to be descheduled.
- M02: the admission check goes on KV occupancy, the resource that actually binds — not on the utilisation metric that saturates.
And in each case the second half matters as much as the first. C01 does not prevent duplicates, it makes them no-ops. C05 does not prevent overload, it makes the failure a countable drop instead of an invisible 18-second queue. C11 does not prevent zombies, it makes their writes harmless. A design that makes the bad case cheap beats one that makes it rare, because rare failures are the ones nobody has tested.