Warmup Guide — Capstone

Orientation for Phase 08. Three capstones — realtime detection, face recognition, medical segmentation — each composing Phases 00–07 into a system with measured properties and a domain-specific hard part. This warmup maps the composition and sets the bar.

Table of Contents

Chapter 1: What the Capstones Certify

Same shift as every track's capstone (the model-accuracy and LLM capstone warmups state it identically — it's one discipline): decisions over implementations, numbers over claims, failure handling over happy paths. The CV-specific addition: each capstone has a domain hard part that the naive port of earlier labs gets wrong — realtime's latency contract, recognition's open-set problem, medical's label-scarcity-plus-stakes. The capstone is passed when the domain hard part is named, measured, and engineered for — not when the demo runs.

Chapter 2: Capstone 01 — Realtime Detection: the Latency Contract

Compose: Phase 05 (YOLO fine-tune) + Phase 01 (tracking) + Phase 07 (serving/perf playbook).

The hard part: "realtime" is a contract, not an adjective — state it as numbers (e.g., 720p @ 25 FPS sustained, p99 end-to-end < 80 ms, on this hardware) and engineer the whole pipeline to it:

  • The budget table is the centerpiece artifact: capture → decode → preprocess → inference → NMS/post → tracking → render, measured (Phase 07 Ch. 8's playbook). Expect decode and pre/post to rival inference; optimize what the table says.
  • Tracking is the realtime multiplier: detector at 10 FPS + light tracker (SORT: Kalman + IoU association — Phase 01 Ch. 7) in between = smooth 30 FPS output with per-object identity; detection-interval vs drift is a tunable you sweep and plot.
  • Throughput mechanics: frame dropping policy (process-latest beats process-all — a queue that backs up is a latency bomb), resolution/model-size operating point (Phase 05 Lab 01's curve, now load-bearing), FP16/INT8 with eval gates.
  • Honest evaluation: sustained-load measurement (thermals throttle — minutes, not seconds), accuracy under the realtime config (the deployed operating point, not the offline best), and the failure demo: what happens at 2× expected object density.

Chapter 3: Capstone 02 — Face Recognition: Open-Set Thinking

Compose: Phase 03/06 (embedding backbone) + Phase 02 (evaluation discipline) + Phase 07 (serving).

The hard part: recognition is not classification — the people at deployment time were not in the training set. The architecture is metric learning: an embedding network maps faces to vectors where same-person distance < different-person distance; "who is this" = nearest enrolled embedding if close enough.

  • The pipeline: detect face → align (landmarks → canonical pose — alignment quality moves accuracy more than model choice; Phase 01 geometry pays off) → embed (a pretrained margin-loss network — ArcFace-class: softmax with an angular margin that forces inter-class separation on the hypersphere; understand the loss even though you won't train it from scratch) → match against the enrolled gallery.
  • The threshold is the product decision: same-person/different-person distance distributions overlap; the threshold sets FAR (false accept) vs FRR (false reject), and the right point depends on the application (unlocking a phone vs flagging a watchlist are opposite asymmetries — Phase 02 Ch. 6's cost-based threshold logic, with teeth). Deliverable: the ROC/DET curve on your gallery + the chosen operating point defended in writing.
  • Open-set evaluation: include impostors (people not enrolled) in every eval; report identification (rank-1) and verification (FAR@FRR) metrics; small galleries flatter — say so.
  • The responsibility chapter: face recognition has documented demographic accuracy disparities (NIST FRVT findings) and real misuse potential. The capstone takes a position: evaluate across demographic slices of your eval data where feasible, document limitations explicitly, and state appropriate-use boundaries in the README. Treating this as integral engineering — not a disclaimer — is part of the bar.

Chapter 4: Capstone 03 — Medical Segmentation: Stakes and Statistics

Compose: Phase 05 (U-Net) + Phase 02 (evaluation/statistics) + Phase 06 (foundation features, optionally) + Phase 07 (reproducibility).

The hard part: small data, expensive expert labels, and consequences — the domain where this curriculum's evaluation ethics either hold or are exposed.

  • Data reality: dozens-to-hundreds of studies, not millions of images. Augmentation is load-bearing (elastic deformations — the U-Net paper's original trick — plus intensity transforms matched to the modality); patient-level splits are non-negotiable (slices from one patient straddling train/test is Phase 02's group leakage at its most flagrant — and the most common published error in the field).
  • Metrics that match the clinic: Dice (overlap) + boundary distance metrics (Hausdorff 95 — a segmentation can have great Dice and clinically unacceptable boundary excursions); per-case distributions, not just means (the worst case is the clinically relevant one — report it); confidence intervals over cases (n is small; Phase 02 Ch. 7's spread, mandatory here).
  • Uncertainty and abstention: a model that flags low-confidence cases for human review is deployable; one that silently segments everything is not. Cheap version: test-time augmentation variance or MC-dropout as an uncertainty proxy, with a flag-rate vs error-caught curve.
  • Reproducibility as ethics: config + seed + data manifest per result (Phase 07 Ch. 6's lineage) — in a domain where results inform care decisions, the ledger is not optional hygiene.

Chapter 5: The Shared Engineering Bar

Identical to the other tracks' capstone bars (one curriculum, one discipline):

  • Walking skeleton first, then deepen; demo exists from day two.
  • Eval plan written before the build — each capstone's metrics, splits, and acceptance numbers declared up front (Ch. 2–4 give the domain-correct metrics).
  • Config-driven, seeded, ledgered: every reported number reproducible from a config file; the results table is append-only.
  • Golden-input tests + preprocessing parity (Phase 07 Ch. 4) in CI for each serving path.
  • One honest limitation paragraph per repo, plus the domain hard part explicitly addressed (latency contract / open-set + responsibility / patient-level stats).
  • The 90-second demo path on a clean machine.

Chapter 6: Portfolio Conversion

  • README order: money table/plot first (the budget table; the DET curve; the per-case Dice distribution), run-it second, design notes third.
  • One rehearsed 2-minute walkthrough per capstone: problem → domain hard part → key decision with evidence → the number → the limitation.
  • Cross-link the lineage: "tracking from Phase 01 Lab 04, operating-point method from Phase 05 Lab 01, parity tests from Phase 07 Lab 02" — showing the curriculum composing is itself the portfolio's story.

Lab Walkthrough Guidance

Sequencing: any order works; realtime detection first is recommended (fastest feedback loop, most reusable infrastructure). Per-capstone steps live in each README; the cross-cutting protocol:

  1. Write the eval plan and the domain-hard-part statement before code.
  2. Skeleton → golden tests → deepen → ledger throughout.
  3. Timebox 1–2 weeks each; cut scope, not corners — a smaller system meeting its stated contract beats a sprawling one meeting none.

Success Criteria

The capstone phase — and the track — is complete when:

  1. Each capstone meets its own declared acceptance numbers, reproducibly from config.
  2. Capstone 01 ships the measured budget table and sustains its FPS/latency contract under load.
  3. Capstone 02 ships the DET/ROC curve with a defended operating point, open-set evaluation, and the responsibility section.
  4. Capstone 03 ships patient-level splits, per-case Dice + HD95 distributions with CIs, and an abstention mechanism with its flag-rate curve.
  5. All three pass the 90-second demo test and carry their limitation paragraphs.

Interview Q&A

Q: Your realtime demo runs 30 FPS on your laptop and 11 FPS on the deployment box. Walk through it. Budget table on the target first (never extrapolate across hardware): usual suspects — no GPU/different GPU (inference 3×), CPU decode bottleneck (different codec/turbojpeg absence), thermal throttling (sustained vs burst — measure minutes in), Python worker count vs cores, and resolution/model mismatch in the deployed config. Fix what the table indicts; then re-state the contract for the actual hardware — perhaps with the smaller model at the measured operating point. Hardware-specific measurement as reflex is the graded behavior.

Q: Why can't you ship a face recognizer trained as a classifier over your user base? Closed-set classification fails structurally: new users require retraining (a softmax head per population), unenrolled people get confidently misassigned (no "none of the above"), and per-user data is tiny. Metric learning fixes all three: enrollment is embedding storage (no retraining), thresholds give a principled reject option, and the embedding generalizes from a large disjoint training population. Recognizing classification-vs-open-set as an architecture decision, not an accuracy tweak, is the point.

Q: Your medical model's mean Dice is 0.91 but the clinician rejects it. What might the mean be hiding? The distribution: a few catastrophic cases (Dice 0.3 on atypical anatomy) inside a high mean — clinical acceptability is governed by the worst case, hence per-case plots and HD95 (boundary excursions Dice can't see). Also: patient-level leakage inflating the number, eval cases unrepresentative of the clinic's scanner/protocol mix, and no abstention path (the clinician needs to know when not to trust it). The answer's spine: means hide tails; medicine lives in the tails.

References