Warmup Guide — Capstone

Orientation for Phase 13. No new theory — the capstones test whether Phases 01–12 compose. This guide covers the integration discipline: walking skeletons, seam testing, the report contract, and converting capstones into interview ammunition.

Table of Contents

Chapter 1: What Changes at Capstone Altitude

A lab asks "can you implement X?"; a capstone asks "do X, Y, and Z work together, and can you prove it with numbers?" Three shifts:

  1. Interfaces over implementations: each phase artifact becomes a component with a contract (the feature store takes events, returns point-in-time training frames; the ranker takes candidates + features, returns ordered ids). Integration failures are contract failures — an undeclared assumption (timestamp semantics, id namespaces, score scales) crossing a seam.
  2. Numbers over claims: every capstone deliverable is a table — offline metrics vs baseline, latency percentiles, experiment readouts. "It works" is not a result; the money table is.
  3. Operations over happy paths: the gates (Phase 08), guardrails (Phase 10), and monitors (Phase 12) are part of the build, not garnish — an injected failure that the system catches is a headline demo.

Chapter 2: The Walking Skeleton

Build the thinnest end-to-end path first: every component present, each trivially implemented (popularity recommender, pass-through guard, constant forecast), one artifact flowing from entry point to report. Then deepen components one at a time, keeping the skeleton green.

Why this beats building components-then-integrating: integration risk — the highest-variance risk — dies in day one instead of week three; the demo exists from the start (and stays demoable through every iteration); and each deepening step has a measurable before/after (the trivial implementation is the baseline your money table needs — a popularity recommender or seasonal-naive forecast is not scaffolding, it's the comparison row).

Chapter 3: Seams — Where Integration Fails

The recurring seam bugs this track has armed you against — check each explicitly:

  • Time semantics (P02/P03): event time vs processing time crossing a boundary; a feature computed with as-of semantics trained on, but served with latest-value semantics — the skew you built detection for. Write the seam test: online and offline features for the same (entity, timestamp) must match.
  • Leakage across stages (P03/P07): the orchestrator (P08) makes data flows explicit — use its lineage to verify nothing downstream of the label enters features.
  • Id and schema drift (P01): every seam gets a data contract; the contracts are the integration documentation.
  • Score-scale assumptions (P05/P06): a ranker consuming retrieval scores must not assume calibration; a canary comparing two models must compare like units.
  • Clock and RNG injection (P01): the moment two components share simulated time (stream + serving simulator), a single injected clock is the difference between deterministic tests and flaky ones.

Each seam check is one test in the capstone's suite — the suite is small but aimed at the seams, because the components already carry their own phase tests.

Chapter 4: The Report Contract

The capstone README leads with results (the same shape every phase lab used):

  1. Money table: headline metrics with baselines — BPR recall@10 0.142 vs popularity 0.031, p99 38 ms at 900 QPS, groundedness 0.94, attack detection 1.00, MASE 0.48 vs seasonal-naive 0.85. Real numbers from your runs; ranges honest.
  2. How to run: the one entry point, expected runtime, what appears.
  3. Architecture sketch: components and seams, one diagram or ASCII block.
  4. Design notes: the 3–4 decisions that mattered, each with the alternative you rejected and why.
  5. Honest limitations: what's simulated (single process, synthetic data), what breaks at scale (the in-memory feature store, the brute-force ANN), and the next-step you'd build. This paragraph reads as senior; its absence reads as not knowing.

Chapter 5: From Capstone to Interview

  • Every system-design interview answer in this track's domain ("design a recommender / feature platform / GenAI assistant / forecasting service") is a guided tour of a capstone you built — rehearse the tour: 2 minutes end-to-end, then dive where the interviewer steers.
  • Resume bullets come from the money table: built X composing Y, measured Z. Never invent scale you didn't run; "simulated at 10k QPS" is honest and still impressive.
  • The behavioral bank (interview-prep/05) wants stories with stakes — the capstone's injected-failure demos (the gate that caught the broken data, the canary that rolled back) are exactly those stories in miniature.

Lab Walkthrough Guidance

For whichever capstone you pick:

  1. Read its README's stage list; copy the relevant phase labs' solutions into a components/ package (they were designed to be composed — same dataclass idioms, injected clocks/RNGs throughout).
  2. Day one: the walking skeleton + the entry point + a trivial money table.
  3. Deepen one component at a time; after each, re-run the seam tests and update the table — the git history should read as a sequence of measured improvements.
  4. Add the injected-failure demo last (broken data caught by contracts; drifted stream caught by PSI; injection caught by canaries; SRM caught by the analyzer) — one per capstone minimum.
  5. Write the limitations paragraph before polishing — it tells you what not to gold-plate.

Success Criteria

The capstone — and the track — is complete when:

  1. One capstone runs end to end from a single entry point on a clean machine, CPU-only, in minutes.
  2. Its money table shows the deepened system beating its own walking-skeleton baseline on the primary metric.
  3. The seam tests pass, and at least one injected failure is demonstrably caught by a gate/guard/monitor.
  4. The README fulfills the report contract including honest limitations.
  5. You can give the 2-minute architecture tour without notes, and answer "why not ?" for each major design decision.

Interview Q&A

Q: Walk me through the hardest integration bug you hit building this. Have a real one from the seam list — time-semantics mismatches and score-scale assumptions are the usual suspects. The answer's shape: the symptom (metrics looked great offline, diverged in the serving path), the localization method (seam test / lineage query / parity check), the fix, and the guard you added so it can't recur. That last clause is what separates senior answers.

Q: Your capstone is synthetic and single-process. Why should I believe it transfers? Because the failure modes it engineers against are scale-independent: point-in- time correctness, training-serving skew, SRM, drift laundering, injection — all occur identically at any scale; the synthetic setup makes them testable. What changes at scale is the infrastructure (distributed stores, real ANN, real streams) — and the components are interface-shaped precisely so those swap in. Then name the two things that genuinely don't transfer (network partitions, organizational data ownership) — the honest boundary is the credibility.

References

  • The track's twelve phase WARMUPs — the actual prerequisite list
  • system-design/ — the five walkthroughs, which are the capstones' design docs written in interview form
  • The Pragmatic Programmer — tracer bullets (the walking skeleton's origin)
  • Hidden Technical Debt in ML Systems — re-read after building; it reads differently now