Phase 07 — Profiling & Performance Engineering
Difficulty: ⭐⭐⭐⭐⭐
Estimated Time: 3 weeks (70–90 hours)
Roles Supported: All senior ML roles — the ability to identify and fix performance bottlenecks is a core Senior Staff competency
Why This Phase Exists
"The model is slow" is not an actionable bug report. "The model's matrix multiplications are compute-bound at 45% utilization and the LayerNorm ops are memory-bound at 12% roofline efficiency" is. Senior Staff engineers don't guess at performance problems — they measure.
This phase teaches the systematic methodology: (1) measure theoretical limits with the roofline model, (2) profile actual execution to find the gap, (3) identify the bottleneck category (memory-bound vs compute-bound vs launch-overhead-bound), (4) apply the correct optimization, (5) verify the fix. Every production ML team at every hardware company does this. At Qualcomm, this applies to both CUDA profiling (R&D) and HTP profiling (product).
Concepts
- Roofline model: performance ceiling = min(peak FLOPS, peak memory bandwidth × arithmetic intensity); ops below the "roofline" are inefficient; ops at the roofline are optimal
- Arithmetic intensity (AI): FLOPS / bytes_transferred; compute-bound ops have high AI (matmul: O(n)/O(n²) = O(n)); memory-bound ops have low AI (elementwise: 1 FLOP / 8 bytes)
- Compute ceiling: peak FLOPS for a device = cores × clock × FLOPs-per-cycle; A100 = 312 TFLOPS FP16
- Memory bandwidth ceiling: peak DRAM bandwidth; A100 HBM = 2 TB/s; Snapdragon 8 Gen 3 DRAM ≈ 77 GB/s
nsys(Nsight Systems): timeline profiler; shows CUDA API calls, kernel launches, data transfers, CPU-GPU synchronization; best for understanding overall execution structurencu(Nsight Compute): per-kernel profiler; shows occupancy, memory throughput, warp stalls, FLOP counts; best for diagnosing individual kernel inefficiencytorch.profiler: PyTorch's built-in profiler; traces op-level timing with CUDA kernels;with torch.profiler.profile(activities=...)context managertorch.utils.benchmark:Timerclass for reliable benchmarking; handles warmup, multi-run statistics, eliminates timing noise- CUDA occupancy: fraction of max concurrent warps scheduled; low occupancy = wasted SM cycles; limited by: register pressure, shared memory usage, block size
- Memory coalescing: adjacent threads accessing adjacent memory → single wide DRAM transaction; non-coalesced access → N separate transactions → N× bandwidth waste
- Warp divergence: threads in the same warp taking different branches → lanes serialized; common cause of GPU inefficiency in sparse or conditional code
- CPU-GPU synchronization overhead: each
tensor.item(),tensor.numpy(), print of GPU tensor causes synchronization; destroys async execution pipeline - HTP profiling: Qualcomm AI Hub provides per-layer timing breakdown; analyze with AI Hub Python API; identify HTP vs CPU fallback ops
Labs
Lab 01 — Roofline Analysis Tool from Scratch
| Field | Value |
|---|---|
| Goal | Build a roofline_analyzer tool that (1) measures a GPU's peak FLOPS and memory bandwidth empirically, (2) instruments a PyTorch model to measure actual FLOP count and memory traffic per op, and (3) plots ops on the roofline diagram with bottleneck classification |
| Concepts | Memory bandwidth measurement (large memcpy benchmark), FLOP counting via torch.profiler, arithmetic intensity per op, roofline visualization, bottleneck classification |
| Steps | 1. Implement measure_peak_bandwidth(): time a large memcpy (e.g., 1 GB) and compute GB/s; 2. Implement measure_peak_flops(): time a large matmul and compute TFLOPS; 3. Implement count_flops_per_op(): use torch.profiler + flop_count_utils to get FLOP count for each op; 4. Implement measure_memory_traffic(): estimate bytes read/written per op from tensor sizes; 5. Implement plot_roofline(): matplotlib scatter plot with compute/memory ceilings; color code ops: red=compute-bound, blue=memory-bound, gray=launch-overhead-bound; 6. Run on: a single matmul (should be compute-bound), LayerNorm (should be memory-bound), full LLaMA-tiny forward pass |
| Stack | PyTorch 2.3+, CUDA GPU, matplotlib, torch.utils.flop_counter |
| Datasets | Synthetic |
| Output | RooflineAnalyzer class with analyze(model, inputs) method; roofline plot as PNG; per-op FLOPS, bandwidth, arithmetic intensity, bottleneck label |
| How to Test | pytest test_lab.py — checks: bandwidth measurement within 10% of spec, matmul is correctly classified as compute-bound, LayerNorm as memory-bound, plot PNG generated |
| Talking Points | "What is the arithmetic intensity of a transformer attention layer?" (depends on batch/seq, explain the crossover point), "How would you improve a memory-bound op?" (tiling, fusion), "What happens to arithmetic intensity at larger batch sizes?" |
| Resume Bullet | Built GPU roofline analysis tool from scratch; identified that 73% of ops in a production transformer are memory-bound at <15% roofline efficiency, leading to 2.1× optimization via kernel fusion |
| Extensions | Add HTP roofline using AI Hub timing data; add memory access pattern visualization; add ETE (Execution Time Estimator) combining roofline per op |
Lab 02 — Custom PyTorch Profiling Harness
| Field | Value |
|---|---|
| Goal | Build a production-quality profiling harness that traces: op-level timing, CUDA kernel timing, memory allocation/free events, and CPU-GPU synchronization points — all without modifying model code |
| Concepts | torch.profiler.profile, ProfilerActivity, CUPTI (CUDA Profiling Tools Interface) traces, torch.profiler.record_function, Chrome trace JSON, torch.cuda.memory_stats(), synchronization detection |
| Steps | 1. Implement ProfilingContext wrapper that enables torch.profiler with CUDA activities; 2. Implement MemoryTracker hook that logs allocation/free per tensor; 3. Implement SyncDetector that patches tensor.item(), tensor.numpy(), cuda.synchronize() to emit warnings; 4. Implement ProfileReport.generate() that produces: top-10 ops by CUDA time, memory peak, sync count, Chrome trace JSON; 5. Run on ResNet-50, BERT-base, and a deliberately bad model (has many .item() calls) |
| Stack | PyTorch 2.3+, torch.profiler, json |
| Datasets | Synthetic |
| Output | ProfilingContext class; Chrome trace JSON loadable in chrome://tracing; console report with top-10 slow ops, memory peak, sync violations |
| How to Test | pytest test_lab.py — checks: sync detector fires on .item() call, memory tracker reports peak within 5% of actual CUDA memory, trace JSON is valid Chrome format |
| Talking Points | "What is the CUPTI overhead of profiling?" (5–10% slowdown), "How do you profile a model in production without the profiler overhead?" (statistical sampling), "What is the typical sync budget for a training step?" |
| Resume Bullet | Built custom profiling harness that detected 47 synchronization violations in a production model; eliminating them reduced step time by 23% |
| Extensions | Add torch.autograd.profiler integration for gradient timing; add distributed profiling for multi-GPU; add NPU trace visualization |
Lab 03 — End-to-End Bottleneck Identification & Optimization
| Field | Value |
|---|---|
| Goal | Apply the full profiling methodology to a deliberately sub-optimal transformer inference implementation: profile → classify bottlenecks → apply targeted optimizations → verify improvements reach 80%+ of theoretical maximum |
| Concepts | KV cache impact, attention memory complexity, torch.compile speedup, operator fusion opportunities, batch size effects on arithmetic intensity, memory layout (contiguous vs non-contiguous tensors) |
| Steps | 1. Start with a naive transformer implementation with 5 seeded performance bugs: (a) no KV cache, (b) non-contiguous attention output, (c) CPU↔GPU sync in generation loop, (d) LayerNorm computed in FP32 even for FP16 model, (e) sequential instead of batched generation; 2. Profile with Lab 02 harness to find all 5; 3. Fix each bug; 4. Apply torch.compile; 5. Compare: buggy vs fixed vs compiled; document speedup for each fix |
| Stack | PyTorch 2.3+, Lab 01 and 02 harnesses |
| Datasets | WikiText-2 (for generation timing) |
| Output | Before/after timing table for each of 5 optimizations; cumulative speedup chart; final inference throughput vs theoretical peak |
| How to Test | pytest test_lab.py — checks: KV cache produces identical output, no sync violations in fixed model, compiled model faster than fixed-only, generation throughput improvement documented |
| Talking Points | Walk through each bug from the interview answer perspective: "I've seen production models with these exact issues. Here's how I found and fixed them." |
| Resume Bullet | Profiled and optimized transformer inference implementation; identified 5 performance bugs via systematic profiling; total 4.7× throughput improvement (KV cache: 2.1×, sync removal: 1.4×, compile: 1.6×) |
| Extensions | Add continuous batching; add FP8 computation; measure the speedup from FlashAttention (preview for Phase 08) |
Deliverables Checklist
- Roofline analyzer with plot for LLaMA-tiny (compute-bound vs memory-bound op classification)
- Profiling harness with Chrome trace output
- Before/after optimization table showing 4× cumulative speedup
-
All
test_lab.pysuites pass
Interview Relevance
- "Our model achieves 30% of peak FLOPS on A100. Where would you start looking?" — you walk through the roofline analysis: arithmetic intensity, memory-bound ops, kernel launch overhead
- "What tools do you use for GPU profiling?" — you describe the tool stack:
nsysfor timeline,ncufor kernel,torch.profilerfor Python-level; explain when to use each - "How do you measure the impact of a performance change reliably?" — explain warmup, multi-run averaging,
torch.utils.benchmark.Timer, preventing benchmarking artifacts