The Week Generator
Weeks 1–12 are written out session by session. Weeks 13–130 are not, and deliberately so: pre-writing 118 weeks two years in advance produces fiction, and fiction you then feel obliged to follow.
What you need instead is the procedure that produced weeks 1–12, so you can run it yourself at the start of every project. That is this page: an algorithm, the invariants a valid decomposition must satisfy, three fully worked examples, and the re-planning procedure for when a week goes wrong.
Time cost: 45 minutes per project, in the last session before it starts. Fourteen times over the journey.
Table of Contents
- Why This Is a Generator and Not a Schedule
- The Algorithm
- The Eight Invariants
- Worked Example 1 — P02, a Medium Project
- Worked Example 2 — P04, With a Mid-Project Pivot
- Worked Example 3 — P05, a Large Project With a Kill Switch
- Decomposition Smells
- Re-Planning Mid-Project
- The Week-Zero Ritual
- References
Why This Is a Generator and Not a Schedule
Three reasons, and the third is the one that matters.
1. Estimates decay. By the time you reach P12 in month 27 you will know your own throughput on this kind of work far better than I do now. A schedule written today encodes my guess; a generator run then encodes your measurement.
2. Projects reshape themselves. P02's milestone 7 (the neighbour-selection heuristic) might take four hours or twelve depending on what milestone 5 taught you. A decomposition made after milestone 5 is strictly better than one made before it.
3. Producing the decomposition is itself the planning skill. Deciding what constitutes a week's worth of work, what can be verified at the end of it, and what depends on what — that is project management at the scale where you actually control the variables. Handing you 118 pre-baked weeks would remove the one form of estimation practice this journey naturally provides. You will be estimating for the rest of your career; here you get fourteen calibrated repetitions with recorded predictions and recorded actuals.
This is the same argument as step 3 of the loop. Write your own design before reading the canonical one. Write your own weeks before reading someone else's.
The Algorithm
Inputs: the project page's milestone table (numbered, with hours), experiments table, readings table, and exit criteria. Output: one row per week with the six weekly outputs.
Step 1 — Check the budget reconciles
Sum the milestone hours. It must equal the project's stated budget.
Σ milestone hours == project budget == weeks × your pace
If it does not, one of them is wrong and you must fix it now, not discover it in week 6. When I first wrote these fifteen project pages, six of them over-allocated their milestone tables by 4–10 hours against their own budgets; a thirty-second script found all six. Run the check.
Step 2 — Normalise milestone sizes
- Any milestone > 12 h is split. It will not fit in a week alongside the week's other obligations, and a milestone spanning three weeks gives you no completion signal for a fortnight.
- Any milestone < 3 h is merged into an adjacent one. Sub-3-hour items are tasks, not milestones, and tracking them adds ceremony without information.
Step 3 — Reserve the non-implementation hours
Your pace is 11 h/week, but implementation is only 45% of it (allocation). The milestone hours are the implementation. So:
\[ \text{weeks} = \frac{\Sigma\,\text{milestone hours}}{11} \quad\text{and each week has}\quad \begin{cases} \sim 5.0\ \text{h implementation (milestones)}\ \sim 1.7\ \text{h reading}\ \sim 2.2\ \text{h experiment}\ \sim 1.1\ \text{h writing}\ \sim 1.1\ \text{h debug/review} \end{cases} \]
This is the step everyone gets wrong, and it is why self-made plans overrun by ~2×. The milestone hours are ~45% of your week, not 100% of it. A 77-hour project is 7 weeks at 11 h/week — not 77/11 = 7 weeks of pure milestone work crammed into 7 weeks.
The project pages already do this arithmetic for you: P02's 77 milestone hours are budgeted as 7 weeks precisely because 77 h ÷ 11 h/wk = 7, with the milestone hours counted as the whole week rather than 45% of it. That is a deliberate simplification — the milestone estimates are generous enough to absorb the reading and experimentation attached to them. If your actuals say otherwise after two projects, re-derive with the 45% rule and lengthen everything.
Step 4 — Pack milestones into weeks
Greedy, in dependency order, filling to ~11 hours per week:
week = 1; budget = 11
for each milestone m in order:
if hours(m) <= budget:
assign m to week; budget -= hours(m)
else if budget >= 3:
split m: put `budget` hours in this week, remainder in the next
else:
week += 1; budget = 11; assign m to week; budget -= hours(m)
Then adjust by hand for the invariants. The greedy pass gets you 80% there in five minutes; the hand adjustment is the other 40 minutes and is where the judgement lives.
Step 5 — Attach reading
Take the project's readings table and attach each item to the milestone that needs it, scheduled for the week before or during that milestone — never front-loaded.
Front-loading reading is the single most common self-planning error in this kind of work. It feels responsible and it destroys step 3 of the loop: read HNSW in week 1 and your own graph design in week 3 is a half-remembered copy.
Step 6 — Attach one experiment per week
Every week gets exactly one, from the project's experiments table, or — early on, before there is anything to measure — a correctness property test.
Order them so that each week's experiment is possible given what exists. Do not save all experiments for the end: the whole point of the 20% experimentation budget is that measurement runs alongside building, so that a wrong design is caught in week 3 rather than week 8.
Step 7 — Name the deliverable and the reflection prompt
For each week: what will exist on disk on Sunday that does not exist on Monday? If you cannot name it in a noun phrase, the week is not well formed.
The reflection prompt should be specific to the week, not "what did I learn". Good prompts: "which of my three predictions about block size was furthest off, and was I wrong about the mechanism or the magnitude?"
Step 8 — Run the invariant check, then commit it
Write the table into notebook/weekly/PLAN-<project>.md and commit it before the
project starts. It is a prediction, and like every other prediction in this journey it
is worth more with a timestamp on it. At the end of the project, diff planned against
actual — that diff is your estimation calibration data.
The Eight Invariants
A decomposition that violates any of these is malformed. Check before committing.
| # | Invariant | Why | How to spot the violation |
|---|---|---|---|
| I1 | Week 1 ends with something running | A first week of pure setup sets the tone that this project is administration. It also delays the first real feedback by 25% of a Medium project | Week 1's deliverable is a noun like "environment" or "repo" rather than a behaviour |
| I2 | A correctness gate precedes the first performance measurement | The track's rule: no performance work while a test is red. If week 3 benchmarks something week 4 tests, you will optimise a bug | The first bench week has no test week before it |
| I3 | Every week has exactly one experiment with a pre-written prediction | Zero means you are building without measuring; two means one gets done badly | A week with an empty experiment cell, or three |
| I4 | Reading is attached to the milestone that needs it | Front-loading destroys the naive-design exercise permanently | Weeks 1–2 contain more than ~2 h of reading |
| I5 | The report is not one week at the end | Writing about work you have forgotten produces a worse report and takes longer. Methods sections are written while doing the method | Only the final week mentions the report |
| I6 | No week is more than ~60% integration | A week with no new mechanism is a smell: either the previous weeks under-delivered, or you have found a way to feel busy | A week whose deliverable is "X now works with Y" and nothing else |
| I7 | The last week is slack + report, not new mechanism | Every project overruns somewhere. A final week already spoken for turns a small overrun into a missed exit criterion | The last week introduces a milestone |
| I8 | Milestones are 3–12 h after normalisation | Larger gives no weekly completion signal; smaller is ceremony | Step 2 was skipped |
I1 and I7 together mean a Medium project has ~6 weeks of real mechanism, not 8. Plan for that and the schedule holds; plan for 8 and it does not.
Worked Example 1 — P02, a Medium Project
Input: P02, 77 hours, 7 weeks (W9–W15), 12 milestones.
Step 1 — reconcile
4+5+4+6+9+10+8+8+5+4+8+6 = 77 ✓ equals the stated budget.
Step 2 — normalise
Largest is m6 at 10 h — under 12, no split needed. Smallest is m1/m3/m10 at 4 h — above 3, no merge. No changes.
Step 3–4 — greedy pack
| Week | Milestones | Hours |
|---|---|---|
| 9 | m1 (4) + m2 (5) | 9 |
| 10 | m3 (4) + m4 (6) | 10 |
| 11 | m5 (9) | 9 |
| 12 | m6 (10) | 10 |
| 13 | m7 (8) | 8 |
| 14 | m8 (8) + m9 (5)? | 13 — over |
| ... |
Week 14 overflows. Hand adjustment: move m9 (persistence, 5 h) to week 15 and pair it with m10 (deletion, 4 h), pushing m11 (sweeps, 8 h) and m12 (report, 6 h) — which is now 14 h in one week and violates I7.
The greedy pass has told us something real: 77 hours over 7 weeks leaves no room for a slack week. Resolution — merge m9 and m10 (both small persistence-adjacent work, 9 h together) into week 14, and let week 15 be sweeps + report with sweeps largely running unattended.
Step 5–7 — the finished plan
| Wk | Objective | Milestones | Reading | Experiment (predict first) | Deliverable |
|---|---|---|---|---|---|
| 9 | Ground truth I can trust | m1 generators + RC, m2 brute force + harness | He, Kumar & Chang (1.5 h) — before generating data | Measure RC at d ∈ {16,64,128,512}; predict each first | Brute force + recall harness + the RC table |
| 10 | A graph that works badly | m3 distance fns, m4 random-graph greedy search | Beyer et al. (1.5 h) | recall@10 vs beam width on a random graph. Predict it at ef=64 | A working, bad index with its failure diagnosed |
| 11 | NSW, and the instrument | m5 NSW insertion + beam search | Malkov 2014 NSW (1.5 h) — after your own insertion rule | efSearch sweep with the distance counter. Predict both factors of the speedup model separately | Recall/QPS curve + the two-factor decomposition |
| 12 | The hierarchy | m6 HNSW layers | Malkov & Yashunin §1–3 (2 h) | HNSW vs NSW at equal distance count. Predict the reduction | A curve dominating week 11's |
| 13 | The heuristic that is a correctness property | m7 Algorithm 4 | Malkov & Yashunin Algorithm 4 (1 h) | E7 clustered vs uniform. Predict: clustered is easier. (It is not — this is the week the project's best result appears) | The recall-ceiling measurement + failure analysis |
| 14 | Make it real | m8 compiled loop, m9 persistence, m10 deletion | ANN-Benchmarks protocol (1.5 h) | Re-measure the crossover against the two-factor model after compiling. Predict where it moves | ns/dist down ≥10×; round-trip persistence |
| 15 | Evidence | m11 sweeps, m12 report | Jégou PQ (2 h, optional) | E12 hnswlib comparison. Predict your factor behind | REPORT.md shipped |
Invariant check
| I1 week 1 runs | ✓ brute force + harness executes on day 3 |
| I2 correctness first | ✓ w9 metric-equivalence tests precede w11's first perf number |
| I3 one experiment/week | ✓ seven weeks, seven experiments |
| I4 reading attached | ✓ Malkov lands w11–13, after the naive design in w10 |
| I5 report distributed | ✓ w13's failure analysis is written that week and becomes a report section |
| I6 integration ≤60% | ✓ w14 is the only integration-heavy week, and it adds the compiled kernel |
| I7 last week slack | ⚠ partially violated — w15 has sweeps and the report |
| I8 sizes 3–12 h | ✓ |
I7 is knowingly violated, and that is recorded rather than hidden. The mitigation: w15's sweeps run unattended, so the wall-clock cost is low even though the hours are allocated. If w13's Algorithm 4 work overruns — likely, since it is the subtlest milestone — the declared cut is m10 deletion, which no downstream project needs. Deciding the cut now, while calm, is worth more than deciding it in week 14 under pressure.
Worked Example 2 — P04, With a Mid-Project Pivot
Input: P04, 99 hours, 9 weeks (W35–W43), 13 milestones:
6,8,6,8,8,6,5,8,10,12,8,8,6 = 99 ✓
This project has a structural feature P02 does not: two compaction strategies (m9 size-tiered, 10 h; m10 leveled, 12 h) where the second is the most likely thing to be cut. The decomposition must make that cut cheap.
| Wk | Objective | Milestones | Reading | Experiment | Deliverable |
|---|---|---|---|---|---|
| 35 | Know my disk | m1 measure the device, m2 WAL | — (measure first) | Sequential vs random, read vs write, block sizes. Predict each; you will be wrong about random reads | Your device's real numbers + a replayable WAL |
| 36 | Durable writes | m3 memtable, m4 SSTable writer | O'Neil §1–3 (2 h) | Torn-tail recovery: truncate the WAL at 20 random offsets | FORMAT.md + a WAL that survives truncation |
| 37 | The read path | m5 SSTable reader, m6 Bloom | Bloom 1970 (0.5 h), Monkey (2 h) | Bloom fpr vs theory, and state your measurement resolution | Point reads across N tables; the fpr table |
| 38 | Deletes and ranges | m7 tombstones, m8 merging iterator | LevelDB source (2 h) | Range scan correctness vs a BTreeMap model | Model-based test over 10⁶ ops |
| 39 | Compaction I | m9 size-tiered | Dong et al. RocksDB (2 h) | Amplification counters under uniform keys. Predict W/R/S from the derivation | Live amplification instrumentation |
| 40 | Compaction II | m10 leveled | Rosenblum & Ousterhout (2 h) | Same counters, leveled. Compare against the derived 31/4/1.10 | Both strategies running |
| 41 | Realistic load | m11 workload generator | — | E2 Zipfian vs uniform. Predict which is faster and why mechanically | Seeded generator, reusable in P05 |
| 42 | Break it | m12 crash hardening | Pillai et al. (0.5 h) | E13 ingest faster than compaction. Characterise the collapse | 50 random kill points, zero acknowledged loss |
| 43 | Evidence | m13 experiments + report | — | E3 the crossover figure | REPORT.md + the crossover plot |
The pivot, planned in advance
Decision point: end of week 40. If leveled compaction is not running by then:
| Cut | m10 leveled. Ship size-tiered only |
| Recovered | 12 h ≈ one full week |
| Cost | E3's crossover figure becomes a comparison against published figures rather than your own |
| What survives | Every other exit criterion. The three amplifications are still measured, just for one strategy |
| What to write | "Leveled compaction not implemented; the read-amplification comparison is therefore against published figures rather than my own." One sentence, in the report, under limitations |
Naming the cut, its trigger week, its cost and its wording in advance is the entire technique. A cut decided at week 42 under pressure is an abandonment; a cut decided at week 35 and executed at week 40 is a plan.
Worked Example 3 — P05, a Large Project With a Kill Switch
Input: P05, 143 hours, 13 weeks (W55–W67), 14
milestones: 10+12+8+12+12+14+14+8+8+8+10+10+12+5 = 143 ✓
Two structural features shape everything: the fault injector is built first (m2, before any distributed feature), and this is the project most likely to overrun in the whole journey.
| Wk | Objective | Milestones | Reading | Experiment | Deliverable |
|---|---|---|---|---|---|
| 55 | Determinism before features | m1 simulated network | Lamport 1978 (2 h) | Same seed → identical message order, proven by hashing the trace | A network you can replay |
| 56 | The injector | m2 fault injector (12 h) | FLP (2 h) | Record a failing schedule, replay it, get the identical failure | faultinjector/ — the most reusable artifact of Stage 3 |
| 57 | Plumbing | m3 node skeleton | Raft §1–4 (3 h) | Per-RPC latency histogram | RPC layer + metrics |
| 58 | Replication, hard-coded leader | m4 single-shard replication | Raft §5.1–5.3 (2 h) | Write latency decomposition: fsync / RTT / apply. Predict which dominates | Writes replicating |
| 59 | The oracle | m5 linearizability checker | Herlihy & Wing (2 h) | Detect a violation you injected on purpose | linchecker/ |
| 60 | Elections | m6 Raft election | Raft §5.2 re-read (1 h) | Exactly one leader per term under partition, asserted globally | Elections under injection |
| 61 | Log replication | m7 Raft log replication | Raft §5.4.2, twice (2 h) | Log Matching asserted after every AppendEntries | matchIndex/nextIndex correct |
| 62 | Safety under restart | m8 persistence | Raft §5.4.2 again | Crash-restart any subset; Leader Completeness holds | Survives crash-restart |
| 63 | Exactly-once effects | m9 client sessions | — | Duplicate every message; assert zero duplicate effects | Idempotent retries |
| 64 | Detection | m10 phi-accrual | Hayashibara (1.5 h) | E4 fixed vs phi-accrual under heavy-tailed delay. Predict the FP reduction | The detection frontier plotted |
| 65 | Membership | m11 membership changes | Raft §6 (1 h) | Add/remove a node with no availability loss | Safe reconfiguration |
| 66 | Catch-up | m12 snapshots | Dynamo (2.5 h) | E11 install-snapshot vs log replay. Where does snapshot win? | A follower recovering from a compacted prefix |
| 67 | Scale + evidence | m13 sharding, m14 report | Spanner (2 h, breadth) | E6 asymmetric partition — the one that finds bugs | REPORT.md with a real bug and its interleaving |
The kill switch
Hard decision point: end of week 65 (week 11 of 13).
If the linearizability checker is not clean across ≥1,000 seeded runs by then:
CUT m11 membership, m12 snapshots, m13 sharding (32 h ≈ 3 weeks)
SHIP single-shard Raft, correct, with the injector and the checker
WRITE "Multi-shard routing, membership changes and snapshots are not implemented.
The system is a correct single-shard replicated log."
Why this is the right cut and not a failure. P05's exit criteria are about correctness under fault injection, not about feature count. A single-shard Raft with zero linearizability violations across 1,000 seeded fault runs, a replayable injector, and a documented real bug fully satisfies every exit criterion except the rebalancing one. A three-shard system that fails the checker satisfies none of them.
And the two artifacts with independent portfolio value — faultinjector/ and
linchecker/ — are both delivered by week 59, four weeks before the decision point. The
plan front-loads the things that survive a cut. That is the design principle for
decomposing any Large project.
Note the reading pattern
Raft §5.4.2 appears in weeks 61 and 62, deliberately twice, and the extended paper is spread across weeks 57–62 rather than read up front. The one subtle safety property in the paper is read immediately before the milestone that implements it and again immediately after — because that is the point at which you can actually tell whether you understood it.
Decomposition Smells
Six patterns that mean the plan is wrong. Each is a real failure I have seen in the examples above or built into these pages on purpose.
| Smell | What it means | Fix |
|---|---|---|
| "Setup" week | Week 1 has no running artifact | Move a small mechanism milestone into week 1, even out of dependency order |
| Reading front-load | 6 h of papers in weeks 1–2 | Redistribute to the milestones that need them. If a paper serves no milestone, it is not yet |
| Experiment clump | All experiments in the final two weeks | You are building blind. Move at least one measurement into every week, even if it is a correctness property |
| Report cliff | The final week is 100% writing | Move the methods section into the weeks that perform the method |
| The 13-hour week | Greedy packing overflowed and you left it | Either merge two small milestones elsewhere and shift, or declare the cut now |
| No named cut | A Large project with no pre-declared scope cut and trigger week | Add one. Every Large project needs a kill switch decided while you are calm |
The experiment clump is the most damaging and the most common, because building feels like progress and measuring feels like a detour. It is also the smell that converts this journey back into an ordinary build log.
Re-Planning Mid-Project
Weeks go wrong. The response is a procedure, not an improvisation.
One week behind
Absorb it. Shift everything right by one week and take it out of the project's slack week (I7). Do not compress a milestone to catch up — compression removes the experiment and the reflection first, which are the two things that make the week worth having done.
Two weeks behind
Cut, do not compress. Go to the project page's
three scope tiers and drop from standard toward
MVI. Record it in EXIT-CRITERIA.md's scope-cuts table the day you decide, with the
trigger and the cost.
Then re-run steps 4–8 of the algorithm on the remaining milestones. It takes fifteen minutes and it produces a plan you believe, which a silently-slipping plan is not.
Three or more weeks behind, or a milestone stalled
This is the two-week stall rule, and it is a written decision among four options: reduce scope, change approach, ask for help, or cut the milestone.
At the project level, one additional question: is the pace wrong rather than the plan? If two consecutive projects have run 30% over, your actual pace is ~8.5 h/week, not 11. Re-derive the whole journey from the duration table and move the end date. A 40-month journey completed beats a 34-month journey abandoned in month nine.
The calibration you get for free
At the end of each project, put planned against actual in the report:
| Week | Planned | Actual | Δ | Why |
|---|---|---|---|---|
| 11 | m5 NSW, 9 h | 13 h | +44% | Beam-search stopping condition took two sessions to get right |
After three projects you will know your personal estimation bias — most people are consistently 20–40% optimistic, and knowing your own multiplier turns every subsequent estimate into a good one. This table is the single most valuable by-product of running the generator, and it does not exist if the weeks were handed to you.
The Week-Zero Ritual
45 minutes, in the last session before a project starts. Fourteen times.
- Reconcile the budget (step 1). 2 min.
- Run the greedy pack (steps 2–4). 10 min.
- Hand-adjust for the invariants (step 8 check). 20 min — this is the real work.
- Attach reading and experiments (steps 5–6). 8 min.
- Name the cut: for any Medium or Large project, which milestone goes first, at which trigger week, and the sentence you will write in the report. 5 min.
- Commit
notebook/weekly/PLAN-<project>.md.
Then, and only then, run scaffold/new-project.sh and start
week one of that project.
Do not skip step 5. A cut named while calm costs five minutes; the same cut decided under pressure in week 11 costs a weekend of rationalising and usually takes the wrong thing.
References
- Brooks, F. P. The Mythical Man-Month, anniversary ed. Addison-Wesley, 1995. Chapter 2 on why projects slip one day at a time, and why adding effort to a late project makes it later — the reason the re-planning procedure cuts scope rather than adding hours.
- Boehm, B. W. Software Engineering Economics. Prentice-Hall, 1981. The cone of uncertainty: estimates made at the start of a project are reliably off by a large factor, and the correct response is re-estimation at checkpoints rather than better initial guessing.
- DeMarco, T. Controlling Software Projects. Yourdon Press, 1982. "You cannot control what you cannot measure" — the argument for the planned-vs-actual table.
- Cohn, M. Agile Estimating and Planning. Prentice Hall, 2005. On relative estimation and on why estimates improve with recorded feedback, which is what the calibration table provides.
- Tetlock, P., Gardner, D. Superforecasting. Crown, 2015. Calibration improves only when predictions are recorded and scored — the same reason notebook sections 1–8 are committed before the run.
- Kahneman, D., Tversky, A. Intuitive Prediction: Biases and Corrective Procedures. 1977. The planning fallacy, and the outside view as its correction — which is exactly what your own planned-vs-actual multiplier becomes after three projects.