The Week Generator

Weeks 1–12 are written out session by session. Weeks 13–130 are not, and deliberately so: pre-writing 118 weeks two years in advance produces fiction, and fiction you then feel obliged to follow.

What you need instead is the procedure that produced weeks 1–12, so you can run it yourself at the start of every project. That is this page: an algorithm, the invariants a valid decomposition must satisfy, three fully worked examples, and the re-planning procedure for when a week goes wrong.

Time cost: 45 minutes per project, in the last session before it starts. Fourteen times over the journey.


Table of Contents


Why This Is a Generator and Not a Schedule

Three reasons, and the third is the one that matters.

1. Estimates decay. By the time you reach P12 in month 27 you will know your own throughput on this kind of work far better than I do now. A schedule written today encodes my guess; a generator run then encodes your measurement.

2. Projects reshape themselves. P02's milestone 7 (the neighbour-selection heuristic) might take four hours or twelve depending on what milestone 5 taught you. A decomposition made after milestone 5 is strictly better than one made before it.

3. Producing the decomposition is itself the planning skill. Deciding what constitutes a week's worth of work, what can be verified at the end of it, and what depends on what — that is project management at the scale where you actually control the variables. Handing you 118 pre-baked weeks would remove the one form of estimation practice this journey naturally provides. You will be estimating for the rest of your career; here you get fourteen calibrated repetitions with recorded predictions and recorded actuals.

This is the same argument as step 3 of the loop. Write your own design before reading the canonical one. Write your own weeks before reading someone else's.


The Algorithm

Inputs: the project page's milestone table (numbered, with hours), experiments table, readings table, and exit criteria. Output: one row per week with the six weekly outputs.

Step 1 — Check the budget reconciles

Sum the milestone hours. It must equal the project's stated budget.

Σ milestone hours  ==  project budget  ==  weeks × your pace

If it does not, one of them is wrong and you must fix it now, not discover it in week 6. When I first wrote these fifteen project pages, six of them over-allocated their milestone tables by 4–10 hours against their own budgets; a thirty-second script found all six. Run the check.

Step 2 — Normalise milestone sizes

  • Any milestone > 12 h is split. It will not fit in a week alongside the week's other obligations, and a milestone spanning three weeks gives you no completion signal for a fortnight.
  • Any milestone < 3 h is merged into an adjacent one. Sub-3-hour items are tasks, not milestones, and tracking them adds ceremony without information.

Step 3 — Reserve the non-implementation hours

Your pace is 11 h/week, but implementation is only 45% of it (allocation). The milestone hours are the implementation. So:

\[ \text{weeks} = \frac{\Sigma\,\text{milestone hours}}{11} \quad\text{and each week has}\quad \begin{cases} \sim 5.0\ \text{h implementation (milestones)}\ \sim 1.7\ \text{h reading}\ \sim 2.2\ \text{h experiment}\ \sim 1.1\ \text{h writing}\ \sim 1.1\ \text{h debug/review} \end{cases} \]

This is the step everyone gets wrong, and it is why self-made plans overrun by ~2×. The milestone hours are ~45% of your week, not 100% of it. A 77-hour project is 7 weeks at 11 h/week — not 77/11 = 7 weeks of pure milestone work crammed into 7 weeks.

The project pages already do this arithmetic for you: P02's 77 milestone hours are budgeted as 7 weeks precisely because 77 h ÷ 11 h/wk = 7, with the milestone hours counted as the whole week rather than 45% of it. That is a deliberate simplification — the milestone estimates are generous enough to absorb the reading and experimentation attached to them. If your actuals say otherwise after two projects, re-derive with the 45% rule and lengthen everything.

Step 4 — Pack milestones into weeks

Greedy, in dependency order, filling to ~11 hours per week:

week = 1; budget = 11
for each milestone m in order:
    if hours(m) <= budget:
        assign m to week; budget -= hours(m)
    else if budget >= 3:
        split m: put `budget` hours in this week, remainder in the next
    else:
        week += 1; budget = 11; assign m to week; budget -= hours(m)

Then adjust by hand for the invariants. The greedy pass gets you 80% there in five minutes; the hand adjustment is the other 40 minutes and is where the judgement lives.

Step 5 — Attach reading

Take the project's readings table and attach each item to the milestone that needs it, scheduled for the week before or during that milestone — never front-loaded.

Front-loading reading is the single most common self-planning error in this kind of work. It feels responsible and it destroys step 3 of the loop: read HNSW in week 1 and your own graph design in week 3 is a half-remembered copy.

Step 6 — Attach one experiment per week

Every week gets exactly one, from the project's experiments table, or — early on, before there is anything to measure — a correctness property test.

Order them so that each week's experiment is possible given what exists. Do not save all experiments for the end: the whole point of the 20% experimentation budget is that measurement runs alongside building, so that a wrong design is caught in week 3 rather than week 8.

Step 7 — Name the deliverable and the reflection prompt

For each week: what will exist on disk on Sunday that does not exist on Monday? If you cannot name it in a noun phrase, the week is not well formed.

The reflection prompt should be specific to the week, not "what did I learn". Good prompts: "which of my three predictions about block size was furthest off, and was I wrong about the mechanism or the magnitude?"

Step 8 — Run the invariant check, then commit it

Write the table into notebook/weekly/PLAN-<project>.md and commit it before the project starts. It is a prediction, and like every other prediction in this journey it is worth more with a timestamp on it. At the end of the project, diff planned against actual — that diff is your estimation calibration data.


The Eight Invariants

A decomposition that violates any of these is malformed. Check before committing.

#InvariantWhyHow to spot the violation
I1Week 1 ends with something runningA first week of pure setup sets the tone that this project is administration. It also delays the first real feedback by 25% of a Medium projectWeek 1's deliverable is a noun like "environment" or "repo" rather than a behaviour
I2A correctness gate precedes the first performance measurementThe track's rule: no performance work while a test is red. If week 3 benchmarks something week 4 tests, you will optimise a bugThe first bench week has no test week before it
I3Every week has exactly one experiment with a pre-written predictionZero means you are building without measuring; two means one gets done badlyA week with an empty experiment cell, or three
I4Reading is attached to the milestone that needs itFront-loading destroys the naive-design exercise permanentlyWeeks 1–2 contain more than ~2 h of reading
I5The report is not one week at the endWriting about work you have forgotten produces a worse report and takes longer. Methods sections are written while doing the methodOnly the final week mentions the report
I6No week is more than ~60% integrationA week with no new mechanism is a smell: either the previous weeks under-delivered, or you have found a way to feel busyA week whose deliverable is "X now works with Y" and nothing else
I7The last week is slack + report, not new mechanismEvery project overruns somewhere. A final week already spoken for turns a small overrun into a missed exit criterionThe last week introduces a milestone
I8Milestones are 3–12 h after normalisationLarger gives no weekly completion signal; smaller is ceremonyStep 2 was skipped

I1 and I7 together mean a Medium project has ~6 weeks of real mechanism, not 8. Plan for that and the schedule holds; plan for 8 and it does not.


Worked Example 1 — P02, a Medium Project

Input: P02, 77 hours, 7 weeks (W9–W15), 12 milestones.

Step 1 — reconcile

4+5+4+6+9+10+8+8+5+4+8+6 = 77 ✓ equals the stated budget.

Step 2 — normalise

Largest is m6 at 10 h — under 12, no split needed. Smallest is m1/m3/m10 at 4 h — above 3, no merge. No changes.

Step 3–4 — greedy pack

WeekMilestonesHours
9m1 (4) + m2 (5)9
10m3 (4) + m4 (6)10
11m5 (9)9
12m6 (10)10
13m7 (8)8
14m8 (8) + m9 (5)?13 — over
...

Week 14 overflows. Hand adjustment: move m9 (persistence, 5 h) to week 15 and pair it with m10 (deletion, 4 h), pushing m11 (sweeps, 8 h) and m12 (report, 6 h) — which is now 14 h in one week and violates I7.

The greedy pass has told us something real: 77 hours over 7 weeks leaves no room for a slack week. Resolution — merge m9 and m10 (both small persistence-adjacent work, 9 h together) into week 14, and let week 15 be sweeps + report with sweeps largely running unattended.

Step 5–7 — the finished plan

WkObjectiveMilestonesReadingExperiment (predict first)Deliverable
9Ground truth I can trustm1 generators + RC, m2 brute force + harnessHe, Kumar & Chang (1.5 h) — before generating dataMeasure RC at d ∈ {16,64,128,512}; predict each firstBrute force + recall harness + the RC table
10A graph that works badlym3 distance fns, m4 random-graph greedy searchBeyer et al. (1.5 h)recall@10 vs beam width on a random graph. Predict it at ef=64A working, bad index with its failure diagnosed
11NSW, and the instrumentm5 NSW insertion + beam searchMalkov 2014 NSW (1.5 h) — after your own insertion ruleefSearch sweep with the distance counter. Predict both factors of the speedup model separatelyRecall/QPS curve + the two-factor decomposition
12The hierarchym6 HNSW layersMalkov & Yashunin §1–3 (2 h)HNSW vs NSW at equal distance count. Predict the reductionA curve dominating week 11's
13The heuristic that is a correctness propertym7 Algorithm 4Malkov & Yashunin Algorithm 4 (1 h)E7 clustered vs uniform. Predict: clustered is easier. (It is not — this is the week the project's best result appears)The recall-ceiling measurement + failure analysis
14Make it realm8 compiled loop, m9 persistence, m10 deletionANN-Benchmarks protocol (1.5 h)Re-measure the crossover against the two-factor model after compiling. Predict where it movesns/dist down ≥10×; round-trip persistence
15Evidencem11 sweeps, m12 reportJégou PQ (2 h, optional)E12 hnswlib comparison. Predict your factor behindREPORT.md shipped

Invariant check

I1 week 1 runs✓ brute force + harness executes on day 3
I2 correctness first✓ w9 metric-equivalence tests precede w11's first perf number
I3 one experiment/week✓ seven weeks, seven experiments
I4 reading attached✓ Malkov lands w11–13, after the naive design in w10
I5 report distributed✓ w13's failure analysis is written that week and becomes a report section
I6 integration ≤60%✓ w14 is the only integration-heavy week, and it adds the compiled kernel
I7 last week slackpartially violated — w15 has sweeps and the report
I8 sizes 3–12 h

I7 is knowingly violated, and that is recorded rather than hidden. The mitigation: w15's sweeps run unattended, so the wall-clock cost is low even though the hours are allocated. If w13's Algorithm 4 work overruns — likely, since it is the subtlest milestone — the declared cut is m10 deletion, which no downstream project needs. Deciding the cut now, while calm, is worth more than deciding it in week 14 under pressure.


Worked Example 2 — P04, With a Mid-Project Pivot

Input: P04, 99 hours, 9 weeks (W35–W43), 13 milestones: 6,8,6,8,8,6,5,8,10,12,8,8,6 = 99

This project has a structural feature P02 does not: two compaction strategies (m9 size-tiered, 10 h; m10 leveled, 12 h) where the second is the most likely thing to be cut. The decomposition must make that cut cheap.

WkObjectiveMilestonesReadingExperimentDeliverable
35Know my diskm1 measure the device, m2 WAL— (measure first)Sequential vs random, read vs write, block sizes. Predict each; you will be wrong about random readsYour device's real numbers + a replayable WAL
36Durable writesm3 memtable, m4 SSTable writerO'Neil §1–3 (2 h)Torn-tail recovery: truncate the WAL at 20 random offsetsFORMAT.md + a WAL that survives truncation
37The read pathm5 SSTable reader, m6 BloomBloom 1970 (0.5 h), Monkey (2 h)Bloom fpr vs theory, and state your measurement resolutionPoint reads across N tables; the fpr table
38Deletes and rangesm7 tombstones, m8 merging iteratorLevelDB source (2 h)Range scan correctness vs a BTreeMap modelModel-based test over 10⁶ ops
39Compaction Im9 size-tieredDong et al. RocksDB (2 h)Amplification counters under uniform keys. Predict W/R/S from the derivationLive amplification instrumentation
40Compaction IIm10 leveledRosenblum & Ousterhout (2 h)Same counters, leveled. Compare against the derived 31/4/1.10Both strategies running
41Realistic loadm11 workload generatorE2 Zipfian vs uniform. Predict which is faster and why mechanicallySeeded generator, reusable in P05
42Break itm12 crash hardeningPillai et al. (0.5 h)E13 ingest faster than compaction. Characterise the collapse50 random kill points, zero acknowledged loss
43Evidencem13 experiments + reportE3 the crossover figureREPORT.md + the crossover plot

The pivot, planned in advance

Decision point: end of week 40. If leveled compaction is not running by then:

Cutm10 leveled. Ship size-tiered only
Recovered12 h ≈ one full week
CostE3's crossover figure becomes a comparison against published figures rather than your own
What survivesEvery other exit criterion. The three amplifications are still measured, just for one strategy
What to write"Leveled compaction not implemented; the read-amplification comparison is therefore against published figures rather than my own." One sentence, in the report, under limitations

Naming the cut, its trigger week, its cost and its wording in advance is the entire technique. A cut decided at week 42 under pressure is an abandonment; a cut decided at week 35 and executed at week 40 is a plan.


Worked Example 3 — P05, a Large Project With a Kill Switch

Input: P05, 143 hours, 13 weeks (W55–W67), 14 milestones: 10+12+8+12+12+14+14+8+8+8+10+10+12+5 = 143

Two structural features shape everything: the fault injector is built first (m2, before any distributed feature), and this is the project most likely to overrun in the whole journey.

WkObjectiveMilestonesReadingExperimentDeliverable
55Determinism before featuresm1 simulated networkLamport 1978 (2 h)Same seed → identical message order, proven by hashing the traceA network you can replay
56The injectorm2 fault injector (12 h)FLP (2 h)Record a failing schedule, replay it, get the identical failurefaultinjector/ — the most reusable artifact of Stage 3
57Plumbingm3 node skeletonRaft §1–4 (3 h)Per-RPC latency histogramRPC layer + metrics
58Replication, hard-coded leaderm4 single-shard replicationRaft §5.1–5.3 (2 h)Write latency decomposition: fsync / RTT / apply. Predict which dominatesWrites replicating
59The oraclem5 linearizability checkerHerlihy & Wing (2 h)Detect a violation you injected on purposelinchecker/
60Electionsm6 Raft electionRaft §5.2 re-read (1 h)Exactly one leader per term under partition, asserted globallyElections under injection
61Log replicationm7 Raft log replicationRaft §5.4.2, twice (2 h)Log Matching asserted after every AppendEntriesmatchIndex/nextIndex correct
62Safety under restartm8 persistenceRaft §5.4.2 againCrash-restart any subset; Leader Completeness holdsSurvives crash-restart
63Exactly-once effectsm9 client sessionsDuplicate every message; assert zero duplicate effectsIdempotent retries
64Detectionm10 phi-accrualHayashibara (1.5 h)E4 fixed vs phi-accrual under heavy-tailed delay. Predict the FP reductionThe detection frontier plotted
65Membershipm11 membership changesRaft §6 (1 h)Add/remove a node with no availability lossSafe reconfiguration
66Catch-upm12 snapshotsDynamo (2.5 h)E11 install-snapshot vs log replay. Where does snapshot win?A follower recovering from a compacted prefix
67Scale + evidencem13 sharding, m14 reportSpanner (2 h, breadth)E6 asymmetric partition — the one that finds bugsREPORT.md with a real bug and its interleaving

The kill switch

Hard decision point: end of week 65 (week 11 of 13).

If the linearizability checker is not clean across ≥1,000 seeded runs by then:

CUT   m11 membership, m12 snapshots, m13 sharding   (32 h ≈ 3 weeks)
SHIP  single-shard Raft, correct, with the injector and the checker
WRITE "Multi-shard routing, membership changes and snapshots are not implemented.
       The system is a correct single-shard replicated log."

Why this is the right cut and not a failure. P05's exit criteria are about correctness under fault injection, not about feature count. A single-shard Raft with zero linearizability violations across 1,000 seeded fault runs, a replayable injector, and a documented real bug fully satisfies every exit criterion except the rebalancing one. A three-shard system that fails the checker satisfies none of them.

And the two artifacts with independent portfolio value — faultinjector/ and linchecker/ — are both delivered by week 59, four weeks before the decision point. The plan front-loads the things that survive a cut. That is the design principle for decomposing any Large project.

Note the reading pattern

Raft §5.4.2 appears in weeks 61 and 62, deliberately twice, and the extended paper is spread across weeks 57–62 rather than read up front. The one subtle safety property in the paper is read immediately before the milestone that implements it and again immediately after — because that is the point at which you can actually tell whether you understood it.


Decomposition Smells

Six patterns that mean the plan is wrong. Each is a real failure I have seen in the examples above or built into these pages on purpose.

SmellWhat it meansFix
"Setup" weekWeek 1 has no running artifactMove a small mechanism milestone into week 1, even out of dependency order
Reading front-load6 h of papers in weeks 1–2Redistribute to the milestones that need them. If a paper serves no milestone, it is not yet
Experiment clumpAll experiments in the final two weeksYou are building blind. Move at least one measurement into every week, even if it is a correctness property
Report cliffThe final week is 100% writingMove the methods section into the weeks that perform the method
The 13-hour weekGreedy packing overflowed and you left itEither merge two small milestones elsewhere and shift, or declare the cut now
No named cutA Large project with no pre-declared scope cut and trigger weekAdd one. Every Large project needs a kill switch decided while you are calm

The experiment clump is the most damaging and the most common, because building feels like progress and measuring feels like a detour. It is also the smell that converts this journey back into an ordinary build log.


Re-Planning Mid-Project

Weeks go wrong. The response is a procedure, not an improvisation.

One week behind

Absorb it. Shift everything right by one week and take it out of the project's slack week (I7). Do not compress a milestone to catch up — compression removes the experiment and the reflection first, which are the two things that make the week worth having done.

Two weeks behind

Cut, do not compress. Go to the project page's three scope tiers and drop from standard toward MVI. Record it in EXIT-CRITERIA.md's scope-cuts table the day you decide, with the trigger and the cost.

Then re-run steps 4–8 of the algorithm on the remaining milestones. It takes fifteen minutes and it produces a plan you believe, which a silently-slipping plan is not.

Three or more weeks behind, or a milestone stalled

This is the two-week stall rule, and it is a written decision among four options: reduce scope, change approach, ask for help, or cut the milestone.

At the project level, one additional question: is the pace wrong rather than the plan? If two consecutive projects have run 30% over, your actual pace is ~8.5 h/week, not 11. Re-derive the whole journey from the duration table and move the end date. A 40-month journey completed beats a 34-month journey abandoned in month nine.

The calibration you get for free

At the end of each project, put planned against actual in the report:

WeekPlannedActualΔWhy
11m5 NSW, 9 h13 h+44%Beam-search stopping condition took two sessions to get right

After three projects you will know your personal estimation bias — most people are consistently 20–40% optimistic, and knowing your own multiplier turns every subsequent estimate into a good one. This table is the single most valuable by-product of running the generator, and it does not exist if the weeks were handed to you.


The Week-Zero Ritual

45 minutes, in the last session before a project starts. Fourteen times.

  1. Reconcile the budget (step 1). 2 min.
  2. Run the greedy pack (steps 2–4). 10 min.
  3. Hand-adjust for the invariants (step 8 check). 20 min — this is the real work.
  4. Attach reading and experiments (steps 5–6). 8 min.
  5. Name the cut: for any Medium or Large project, which milestone goes first, at which trigger week, and the sentence you will write in the report. 5 min.
  6. Commit notebook/weekly/PLAN-<project>.md.

Then, and only then, run scaffold/new-project.sh and start week one of that project.

Do not skip step 5. A cut named while calm costs five minutes; the same cut decided under pressure in week 11 costs a weekend of rationalising and usually takes the wrong thing.


References

  • Brooks, F. P. The Mythical Man-Month, anniversary ed. Addison-Wesley, 1995. Chapter 2 on why projects slip one day at a time, and why adding effort to a late project makes it later — the reason the re-planning procedure cuts scope rather than adding hours.
  • Boehm, B. W. Software Engineering Economics. Prentice-Hall, 1981. The cone of uncertainty: estimates made at the start of a project are reliably off by a large factor, and the correct response is re-estimation at checkpoints rather than better initial guessing.
  • DeMarco, T. Controlling Software Projects. Yourdon Press, 1982. "You cannot control what you cannot measure" — the argument for the planned-vs-actual table.
  • Cohn, M. Agile Estimating and Planning. Prentice Hall, 2005. On relative estimation and on why estimates improve with recorded feedback, which is what the calibration table provides.
  • Tetlock, P., Gardner, D. Superforecasting. Crown, 2015. Calibration improves only when predictions are recorded and scored — the same reason notebook sections 1–8 are committed before the run.
  • Kahneman, D., Tversky, A. Intuitive Prediction: Biases and Corrective Procedures. 1977. The planning fallacy, and the outside view as its correction — which is exactly what your own planned-vs-actual multiplier becomes after three projects.