JD2 Labs — Hands-On Inference Optimization
Six labs that turn the knowledge modules into demonstrable, résumé-grade skill. Each lab has: a goal, a stack, numbered steps, a measurable result (a table or curve you produce), and a résumé bullet. Do them in order; each builds on the prior.
| Lab | Produces | Knowledge prereq |
|---|---|---|
| 01 — Quantize & Benchmark | FP16/INT8/INT4 latency·throughput·accuracy table + Pareto plot | K02, K06 |
| 02 — Port PyTorch→ONNX→TensorRT | A working engine + a log of every op that broke and the fix | K03 |
| 03 — vLLM Throughput Tuning | A throughput-vs-p95-latency goodput curve hitting an SLO | K04, K01 |
| 04 — Profile & Find the Bottleneck | An nsys trace + written root-cause of a deliberate bottleneck | K06, K00 |
| 05 — Size a Deployment | A one-page sizing memo: #GPUs, parallelism, $/token, TCO | K08, K05 |
| 06 — Spec-Decode & Disagg Simulator | Speculative-decoding speedup grids (incl. the utilization kill-switch) + colocated-vs-disaggregated goodput tables with the crossover called out | K01 §9, K08 |
Hardware notes
- Labs 01, 03, 04 want a CUDA GPU. No GPU? Use a cloud instance (a single A10/L4/A100 hour is enough), or run CPU/
llama.cppvariants for the concepts (you lose the GPU-specific profiling). - Lab 02 wants NVIDIA + TensorRT; the ONNX export half runs anywhere.
- Lab 05 is pure analysis — no hardware needed, just the math. Do it even with no GPU; it's the most interview-relevant.
- Lab 06 is pure stdlib Python — a deterministic analytic simulator (no GPU, no network, no deps beyond pytest). Runs on any laptop.
The meta-skill
Every lab ends with honest numbers and a written conclusion, because the JD's deliverable is always a defensible report (K06 §7, K08 §9). A lab without a measured result you can defend isn't done.