Warmup Guide — Capstone

Orientation for Phase 11. The capstones contain no new theory — they test whether Phases 01–10 compose. This warmup covers the integration skills the labs assume: pipeline architecture, configuration discipline, debugging across stage boundaries, and turning capstones into portfolio artifacts.

Table of Contents

• Chapter 1: What Changes at the Capstone Level
• Chapter 2: Pipeline Architecture — Stages, Artifacts, Gates
• Chapter 3: Configuration and Reproducibility Discipline
• Chapter 4: Cross-Stage Debugging — The Bisection Discipline
• Chapter 5: The Five Capstones and What Each Certifies
• Chapter 6: From Capstone to Portfolio
• Lab Walkthrough Guidance
• Success Criteria
• Interview Q&A
• References

Chapter 1: What Changes at the Capstone Level

A lab asks "can you implement X?"; a capstone asks "can you make X, Y, and Z work together and prove it with numbers?" Three shifts:

1. Interfaces over implementations: each prior phase's deliverable becomes a component with a contract (the quantizer takes a model + calibration data, returns a model + a report). Capstone failures are nearly always at interfaces — implicit assumptions (layout, dtype, tokenizer version) crossing a boundary uninspected.
2. Numbers over claims: every capstone deliverable is a table or plot — accuracy before/after, latency distributions, Pareto fronts. "It works" is not a result.
3. Failure handling over happy path: pipelines run unattended (CI, sweeps); a stage that crashes without a diagnostic artifact wastes a night of compute. Production thinking starts here.

Chapter 2: Pipeline Architecture — Stages, Artifacts, Gates

The architecture all five capstones share (and that real deployment pipelines — Phase 06 Ch. 4 — share too):

[stage] → artifact + report → [gate] → [stage] → ...

• Stages are pure-ish functions of (input artifact, config): exportable, re-runnable, individually testable. No stage reaches into another's internals.
• Artifacts are files (ONNX, quantized checkpoint, eval JSON, profile trace) with content-addressed or versioned names. If a stage's output isn't a savable artifact, you can't cache, diff, or bisect it.
• Reports travel with artifacts: every stage emits machine-readable metrics (node counts, accuracy, latency) — the regression CI (capstone 01) is just gates over these reports.
• Gates are explicit: numeric thresholds with statistical backing (Phase 09 Ch. 8), not eyeballs. A pipeline without gates silently ships regressions; a pipeline with uncalibrated gates cries wolf until ignored.

This shape — not any specific tool — is what "production quantization pipeline" (capstone 04) means.

Chapter 3: Configuration and Reproducibility Discipline

Sweeps (capstone 04/05 run dozens of configs) die without discipline:

• One config object per run, serialized alongside artifacts. Every knob — bit-width, group size, calibration set hash, seed, library versions — in the config, nothing ambient. The test: a colleague reruns from the config file alone and matches your numbers.
• Seeds are config: calibration sampling, eval subset selection, generation — all seeded. Run-to-run σ measured once (Phase 09 Ch. 8) and quoted with every comparison.
• Naming: {model}-{method}-{bits}g{group}-{calib_hash[:8]} beats final_v2_fixed. Six weeks later, the filename is the documentation.
• The results table is append-only: a CSV/JSON-lines ledger every run appends to — Pareto plots (capstone 05) regenerate from the ledger, never from memory.

Chapter 4: Cross-Stage Debugging — The Bisection Discipline

The capstone debugging situation: end-to-end accuracy is wrong, five stages could be responsible. The procedure (generalizing Phase 05 Ch. 4 and Phase 06's pipeline):

1. Freeze inputs: one fixed batch, fixed seed, through every stage.
2. Bisect over stages: compare each stage's output against the previous stage's under the same inputs (PyTorch vs ONNX, ONNX vs quantized, simulator vs device). The first diverging stage owns the bug.
3. Bisect within the stage — per layer: dump intermediate activations (hooks on one side, per-node outputs on the other); find the first diverging layer. Divergence metrics: max-abs and cosine — max-abs catches outliers, cosine catches systematic rotation; report both.
4. Distinguish expected from broken divergence: quantization legitimately diverges (bounded by step size, Phase 03 Ch. 3); a growing-with-depth divergence is error accumulation; a step-function divergence at one layer is a bug in that layer's transform.
5. Convert every found bug into a gate: the regression suite grows from the pipeline's own pathology. That feedback loop is the difference between a script and infrastructure.

Chapter 5: The Five Capstones and What Each Certifies

Do them in an order that matches your target role's emphasis: NPU-focused → 02, 03 first; accuracy/eval-focused → 01, 05 first; everyone finishes with 04 (it forces the most integration).

Chapter 6: From Capstone to Portfolio

These capstones are interview ammunition only if they are legible:

• README with the money table first: the result (accuracy/latency table, Pareto plot) above the fold; how-to-run second; design notes third.
• One honest limitation paragraph each — "the NPU codegen targets a mock ISA; real-target port would need X" reads as senior; silence reads as not knowing.
• A 90-second demo path: make demo or one notebook that runs on CPU and produces the headline artifact. Interviewers will not install CUDA for you.
• Resume bullets follow the labs' format: implemented X composing Y, measured Z. Numbers from your ledger, ranges honest.

Lab Walkthrough Guidance

Each capstone's README carries its own steps; the cross-cutting advice:

1. Read the target capstone's upstream labs' solutions first — the capstone assumes their interfaces; re-deriving them mid-capstone doubles the time.
2. Build the walking skeleton before any depth: all stages connected with trivial implementations and one artifact flowing end-to-end, then deepen stages. Integration risk dies first; the demo exists from day one.
3. Write the gate tests before the optimization (capstones 01, 04): a known-bad injection that must fail, a known-good baseline that must pass — then develop inside that harness.
4. Keep the ledger from run #1 (Ch. 3) — retrofitting reproducibility is 10× the cost of starting with it.

Success Criteria

The capstones are done when:

1. Each produces its headline artifact from a single entry point, reproducibly from config.
2. The regression CI (01) demonstrably catches an injected 2% degradation and passes a clean change.
3. The graph-optimization capstone (02) shows a before/after profile with the win attributed to named passes.
4. The backend (03) compiles a LLaMA-style block with >95% op coverage and falls back gracefully on the rest.
5. The quant suite (04) reproduces the Phase 03/10 method ranking on at least one model, from one CLI.
6. The dashboard (05) renders the ledger as a Pareto front a non-engineer could choose from.
7. Every repo passes the 90-second demo test on a clean machine.

Interview Q&A

Q: Walk me through your pipeline's design. Why stages and artifacts? Cacheability (re-run only what changed), bisectability (Ch. 4's procedure requires inspectable boundaries), testability (each stage has contract tests), and parallelism (sweeps fan out per config). The alternative — one script with globals — works once and debugs never.

Q: How do you know your pipeline's numbers are right? Three layers: harness validated against an external reference (Phase 09's 0.5% reconciliation); golden-input regression tests on every stage; and the injection test — the pipeline has demonstrably caught a planted error. Numbers from an unvalidated pipeline are decoration.

Q: You ran 60 configs; the best one looks too good. What now? Multiple-comparisons suspicion (Phase 09 Ch. 6): with 60 draws, the max is biased upward. Re-run the winner with fresh seeds and a held-out eval subset; check its neighbors in config space (a real effect has a smooth neighborhood, a fluke is isolated); only then promote it to the baseline.

Q: What would you do differently with another month? Have a real answer per capstone — the honest-limitation paragraphs (Ch. 6) are this question pre-answered. Generic humility scores zero; "the backend's memory planner is first-fit, and fragmentation costs ~8% TCM utilization on the LLaMA block — I'd implement offset-based planning next" scores.

References

• Hidden Technical Debt in Machine Learning Systems (NeurIPS 2015) — why pipelines rot
• Sculley et al., ML: The High-Interest Credit Card of Technical Debt (2014)
• MLflow tracking concepts — the ledger pattern, industrialized
• DVC — artifact versioning patterns worth stealing even without the tool
• The Pragmatic Programmer — tracer bullets (= walking skeleton)
• Phases 01–10 WARMUP and HITCHHIKERS guides — the actual prerequisite list