Warmup Guide — Capstone

Orientation for Phase 11. Nine capstones, one test: do Phases 01–10 compose into systems you can defend with numbers? This warmup maps the capstones to their prerequisite chains, sets the engineering bar, and tells you how to choose.

Table of Contents

• Chapter 1: What the Capstones Certify
• Chapter 2: The Capstone Map — Prerequisites and Payoffs
• Chapter 3: Choosing Your Sequence
• Chapter 4: The Engineering Bar
• Chapter 5: Evaluation Is the Deliverable
• Chapter 6: Portfolio and Interview Conversion
• Lab Walkthrough Guidance
• Success Criteria
• Interview Q&A
• References

Chapter 1: What the Capstones Certify

A lab proves you can build a component against a spec; a capstone proves you can make decisions — scope, architecture, eval design, what to cut — and land a working system with measured properties. The shift in posture (same as the model-accuracy track's capstone warmup, which shares this chapter's DNA): interfaces over implementations, numbers over claims, failure handling over happy paths. Plus one this track adds: cost discipline — every capstone has a compute budget, and engineering under a budget (smaller models, fewer tokens, cheaper evals, honest extrapolation) is itself the skill being certified. "I trained the mini-GPT for exactly the tokens Chinchilla suggests for its size, and here's the loss it predicted vs achieved" is a stronger artifact than an over-trained model with no analysis.

Chapter 2: The Capstone Map — Prerequisites and Payoffs

Read the chain column honestly: a capstone whose chain contains a phase you rushed is where that debt comes due — re-read that phase's WARMUP success criteria first.

Chapter 3: Choosing Your Sequence

Nine is a menu, not a syllabus. Choose 3–4 by target role:

• Inference/serving roles (this track's title): 05 (mini-vLLM) is the centerpiece — build it; then 03 (gateway) and 09 (edge). This trio answers every serving interview.
• Applied/product LLM roles: 02 (RAG) + 07 (agent) + 04 (SFT/DPO) — the retrieval-augmented assistant lifecycle.
• Training-leaning roles: 01 (pretraining) + 08 (RLHF) + 04.
• Sequencing rules: do a serving one and a training one regardless of focus (the perspectives cross-pollinate: you serve what you train and train what you serve); do 05 before 03 if doing both (the gateway is better when you know what's under it); and 09 last if pursuing the model-accuracy track too — it's the bridge artifact.

Chapter 4: The Engineering Bar

What distinguishes a capstone repo from a lab solution (the checklist your reviewers — real or imagined — will apply):

• Walking skeleton first: end-to-end with stub components on day one; deepen inside a working system. Integration risk dies first, demos exist always.
• Config-driven and reproducible: one config object per run, serialized with artifacts; seeds fixed; the ledger (results table) append-only from run #1. (The model-accuracy capstone warmup's Ch. 3 is the full discipline; it applies verbatim.)
• Failure handling is scoped, not skipped: a gateway handles model-backend timeouts; an agent caps iterations; a RAG pipeline degrades to "not found" — each capstone's README states which failures are handled and which are explicitly out of scope. Stating non-goals is the senior move (the PMC track's design-doc lesson).
• Tests at the contracts: the KV-cache equivalence test, the tokenizer round-trip, retrieval recall against brute force, the masking-correctness test — every capstone inherits its chain's signature tests; CI runs them.
• One honest limitation paragraph per repo — what breaks at 10×, what you'd do next, what's mocked.

Chapter 5: Evaluation Is the Deliverable

The single most common capstone failure: building the system and bolting on a demo. Inverted, correctly: define the eval before the build (Phase 08's discipline, applied to yourself):

• 01: loss-vs-budget curve against scaling-law prediction; sample quality progression.
• 02: the five-row ablation (BM25 / dense / hybrid / +rerank / oracle) + faithfulness.
• 03: TTFT/TPOT p50/p99 under Poisson load, goodput at SLO — with the load generator in the repo.
• 04: paired before/after eval with a calibrated judge + the both-directions refusal check.
• 05: throughput-vs-batch curves, KV utilization vs static baseline, equivalence tests.
• 07: task success rate over a fixed task suite, with iteration counts and injection-canary obedience.
• Each number reported with its uncertainty and measurement conditions — the Phase 08 posture, now about your own work.

Chapter 6: Portfolio and Interview Conversion

• README structure: money table/plot first, run-it-in-90-seconds second, design notes third (same as the model-accuracy track — consistency across your portfolio itself reads as discipline).
• Each capstone yields one resume bullet with a measured claim and one 2-minute walkthrough you can deliver aloud: problem → key design choice → the number → the limitation. Practice the walkthrough; the repo is evidence, the narration is the interview.
• Cross-link: your mini-vLLM README should cite its KV-cache lab ancestry and the vLLM paper's mechanisms it implements vs omits — situating your work in the real systems' design space is what makes it read as engineering rather than homework.

Lab Walkthrough Guidance

Per-capstone steps live in each capstone's README; the cross-cutting protocol:

1. Re-read the chain's WARMUP success criteria (Ch. 2's table); patch gaps first.
2. Write the eval plan (Ch. 5) and the non-goals list before code.
3. Walking skeleton → contract tests → deepen → ledger throughout.
4. Timebox: 1–2 weeks per capstone of honest part-time work; a capstone that sprawls past 3 weeks needs its scope cut, not its hours raised (scope discipline is part of the certification).

Success Criteria

The capstone phase — and the track — is complete when:

1. Three-plus capstones pass their own eval plans with documented numbers.
2. Every repo passes the 90-second clean-machine demo test.
3. Signature contract tests run in CI on each (and you can name each test's lineage).
4. Each has its limitation paragraph and its rehearsed 2-minute walkthrough.
5. You can answer "what would you do with 10× the budget" per capstone with a specific, ordered plan — the question that separates builders from operators of tutorials.

Interview Q&A

Q: Walk me through the hardest design decision in your capstone. Have one per repo, structured: the constraint that forced a choice, the 2–3 options with real costs, the evidence that picked the winner, and what you'd revisit. Example shape (mini-vLLM): "block size 16 vs 64 — small blocks cut internal fragmentation (measured 11% → 3%) but doubled block-table overhead per attention call; I chose 16 and fused the table walk into the gather; at 10× scale I'd revisit because table memory itself starts to matter." Decision-evidence-revisit is the senior cadence.

Q: Your numbers look good — how do I know they're not benchmarketing? Invite the audit: conditions stated (hardware, load process, prompt/output distributions), uncertainty reported, baselines included (and strong — BM25, static batching, FP16), load generator and eval sets in the repo, and the limitation paragraph pre-empting the gotchas. The capstone's credibility is the reproducibility kit — which is also the honest answer's last line: "run it."

References

• Each capstone README's own reference list — the chain's papers
• The model-accuracy track's capstone WARMUP — the pipeline/ledger/bisection disciplines, shared
• vLLM, SGLang, llama.cpp — the real systems to situate against
• Karpathy, nanoGPT and llm.c — the pretraining-capstone gold standards for scope discipline
• The Pragmatic Programmer — tracer bullets; Designing Data-Intensive Applications — for the gateway/serving capstones' systems vocabulary