Warmup Guide — Profiling & Performance Engineering

Zero-to-expert primer for Phase 07. Builds performance analysis from "what limits speed" through the roofline model, profiler mechanics, and the taxonomy of transformer performance bugs.

Table of Contents

• Chapter 1: The Two Ceilings — Compute and Bandwidth
• Chapter 2: Arithmetic Intensity
• Chapter 3: The Roofline Model
• Chapter 4: FLOP and Byte Counting for Real Models
• Chapter 5: Prefill vs Decode — The Two Regimes of LLM Inference
• Chapter 6: How Profilers Work
• Chapter 7: Measurement Methodology — How Not to Lie to Yourself
• Chapter 8: The Transformer Performance-Bug Taxonomy
• Lab Walkthrough Guidance
• Success Criteria
• Interview Q&A
• References

Chapter 1: The Two Ceilings — Compute and Bandwidth

Zero background: any processor has two hard limits:

• Peak compute: how many floating/integer operations per second the ALUs can retire (e.g., A100: 312 TFLOPS FP16; a phone NPU: ~10–45 TOPS INT8).
• Peak memory bandwidth: how many bytes per second DRAM can deliver (A100: ~2 TB/s HBM; a phone: ~50–100 GB/s LPDDR).

Every kernel needs both: bytes in, operations on them, bytes out. Whichever resource the kernel exhausts first is its bound. A bandwidth-bound kernel cannot be sped up by faster ALUs, more cores, or lower precision compute — only by moving fewer bytes. The single most common performance-engineering error is optimizing the resource that wasn't the bottleneck; the whole point of this phase is to identify the bound before acting.

Chapter 2: Arithmetic Intensity

Definition: for a kernel,

$$I = \frac{\text{FLOPs performed}}{\text{bytes moved to/from DRAM}} \quad [\text{FLOP/byte}]$$

The machine balance point: a device with peak compute $P$ and bandwidth $B$ has ridge point $I^* = P/B$. If $I < I^$, the kernel is bandwidth-bound; if $I > I^$, compute-bound. A100 FP16: $312,\text{TFLOPS} / 2,\text{TB/s} \approx 156$ FLOP/byte — a high bar.

Canonical intensities (FP16, derive these yourself once):

• Elementwise add ($z = x + y$, n elements): 1 FLOP per element; 6 bytes (read 2, write 1, FP16) → $I = 1/6 \approx 0.17$. Hopelessly bandwidth-bound — runs at ~0.1% of compute peak. This is why fusion (Phase 04 Ch. 9) matters.
• Matrix multiply ($M{\times}K \cdot K{\times}N$): $2MNK$ FLOPs; $2(MK + KN + MN)$ bytes ideally. For large square matrices $I \approx N/3$ — at N=1024, ~341 FLOP/byte: compute-bound. For matrix-vector (M=1): $I \approx 1$ — brutally bandwidth-bound. Same op family, opposite regimes — batch size is destiny.
• Attention (unfused): the $QK^\top$ and $AV$ matmuls plus a softmax over an $n \times n$ matrix that must round-trip DRAM — intensity collapses, which is exactly what FlashAttention fixes (Phase 08).

Chapter 3: The Roofline Model

Plot attainable performance vs intensity (both axes log):

$$\text{Attainable FLOPS}(I) = \min(P,; B \cdot I)$$

The slanted part ($B \cdot I$) is the bandwidth roof; the flat part ($P$) the compute roof; they meet at the ridge $I^*$. Place each kernel at its measured (I, FLOPS) point:

• On/near the slanted roof → bandwidth-bound and running as fast as physics allows: stop optimizing the kernel, raise its intensity (fuse, batch, quantize the bytes).
• On the flat roof → compute-bound and efficient: lower precision or better hardware.
• Below both roofs → neither resource is saturated: launch overhead, poor tiling, serialization, low occupancy — implementation problems. This is the interesting region and where most real kernels live.

Lab 01 builds exactly this: measure device ceilings empirically (a big matmul for P, a big memcpy-like op for B), count FLOPs/bytes per layer, and produce the plot with each layer placed. Refinement to know: multiple ceilings (e.g., without tensor cores, without FP16) give a ladder of roofs; cache-aware rooflines use bytes-from-each-level.

Chapter 4: FLOP and Byte Counting for Real Models

FLOP rules of thumb (memorize):

• Linear layer: $2 \cdot d_{in} \cdot d_{out}$ per token ("multiply + add" = 2).
• Transformer forward: $\approx 2 \cdot N_{params}$ FLOPs per token (every parameter is used in one multiply-add). Attention adds $\approx 2 n^2 d$ per layer for context $n$.
• Training step: $\approx 6 \cdot N_{params}$ per token (forward + ~2× for backward).

Byte counting is subtler and regime-dependent: weights are read once per forward (per batch — they're shared across the batch, the key fact); activations scale with batch × sequence; the KV cache (Phase 02 Ch. 3) is read in full per decoded token. Counting bytes honestly per layer — including intermediate tensors that round-trip DRAM in unfused chains — is the part of Lab 01 that changes how you see models.

Chapter 5: Prefill vs Decode — The Two Regimes of LLM Inference

The most consequential roofline application in modern ML:

• Prefill (process the prompt): all prompt tokens at once → big matmuls (batch × seq rows) → high intensity → compute-bound. Measured in tokens/sec throughput; benefits from FP16/INT8 compute.
• Decode (generate token-by-token): one token's matvecs against all weights + full KV-cache read → $I \approx 1$ → bandwidth-bound. Latency per token ≈ (weight bytes + KV bytes) / bandwidth. Benefits from weight quantization (fewer bytes), GQA (smaller KV), speculative decoding (amortize reads over multiple tokens — Phase 08), and batching (share the weight read across requests — continuous batching's whole premise).

One model, two opposite bottlenecks, two disjoint optimization menus. Every "why is my LLM slow" conversation starts by asking which regime dominates the workload.

Chapter 6: How Profilers Work

Know your instruments' mechanisms, because each lies differently:

• Sampling profilers (py-spy, perf, async-profiler): interrupt periodically, record stacks. Low overhead, statistical; misses short events, great for "where does time go overall".
• Tracing/instrumenting profilers (torch.profiler, Perfetto): record begin/end of every annotated event. Exact spans; overhead distorts very small ops.
• The GPU async trap: CUDA kernels launch asynchronously — Python-side timing measures launch, not execution. torch.profiler correlates GPU events properly via CUPTI; manual timing must use torch.cuda.Event or synchronize() around the region.
• Hardware counter tools (Nsight Compute, vendor NPU profilers): per-kernel utilization, achieved bandwidth, occupancy — the data the roofline needs. Heavyweight; use on the top-5 kernels the trace identified, not everything.

Reading a torch trace (Lab 02): find the gaps (GPU idle = launch overhead or CPU preprocessing on the critical path), the long bars (the kernels to roofline), and the memcpy bars (unintended host↔device transfers — a top-3 real-world finding, usually .item()/.cpu() in the loop).

Chapter 7: Measurement Methodology — How Not to Lie to Yourself

The discipline (every number you report passes these):

1. Warmup before measuring: JIT compilation, cache fill, clock ramp all pollute early iterations (cf. JVM warmup — same physics, different stack).
2. Synchronize at boundaries (GPU async trap above).
3. Report distributions, not means: median + p95/p99 over ≥30 iterations; latency distributions are skewed and tails are what users feel.
4. Control the environment: fixed clocks where possible (nvidia-smi -lgc), thermal steady-state for mobile (Phase 06), no competing processes, pinned versions. Record it all — an unreproducible benchmark is an anecdote.
5. Change one variable per experiment and keep a results table. "I changed three things and it's faster" teaches nothing and regresses later.
6. Validate the workload is real: microbenchmark shapes must match deployment shapes — a kernel tuned for 4096² that deploys at batch-1 decode was tuned for the wrong regime (Chapter 5).

Chapter 8: The Transformer Performance-Bug Taxonomy

Lab 03 plants these in a working model; learn the signatures:

The meta-skill: each bug is invisible in code review and obvious in a trace — measure first is not a slogan, it's the procedure.

Lab Walkthrough Guidance

Order: Lab 01 (roofline) → Lab 02 (profiling context) → Lab 03 (bug hunt) — theory instrument, then the recording instrument, then the applied hunt.

• Lab 01: measure ceilings empirically before trusting datasheets (real sustained bandwidth is 60–80% of nominal); hand-derive the elementwise and matvec intensities from Chapter 2 and confirm your tool reproduces them; then place a small transformer's layers and write one sentence per layer naming its bound.
• Lab 02: build the profiling context manager; verify it against torch.profiler on the same workload; make the GPU-sync handling explicit and test it (the async trap is the lab's real content).
• Lab 03: for each planted bug — find it in the trace first, name the taxonomy row, predict the speedup from fixing, fix, measure, compare to prediction. The predict-then-measure loop is what makes this training rather than tinkering.

Success Criteria

You are ready for Phase 08 when you can, from memory:

1. Derive arithmetic intensity for elementwise, square matmul, and matvec, and place each on a roofline.
2. Compute a device's ridge point from its datasheet and state what it means.
3. Explain prefill vs decode bounds and give two optimizations per regime that don't help the other.
4. State the GPU async-timing trap and both correct measurement methods.
5. Recite five rows of the bug taxonomy with signatures.
6. Estimate tokens/sec for batch-1 decode of a quantized model from bandwidth alone (Phase 03 Ch. 2's formula) — and say when that estimate breaks (compute-bound prefill, small models where overhead dominates).

Interview Q&A

Q: A kernel sits well below both roofline ceilings. What are the candidate causes? Neither memory nor compute is saturated → the kernel isn't issuing enough work: launch overhead dominating (too-small grid), low occupancy (register/shared-memory limits capping resident warps), serialization (dependent small ops, host sync), divergent control flow, or non-coalesced access patterns wasting transactions (bytes requested exceed bytes used — effective bandwidth low even though DRAM is busy). Profile counters (achieved occupancy, memory efficiency) distinguish them.

Q: Quantizing weights to INT4 doubled decode speed but prefill barely moved. Explain. Decode is bandwidth-bound on weight bytes — 4× fewer bytes ≈ proportional speedup (Chapter 5). Prefill is compute-bound; INT4 weights still dequantize to FP16 for the matmul (or the INT4 compute path wasn't faster), so the compute roof didn't move. Fully expected; report regimes separately.

Q: How would you set up performance regression detection for a model in CI? Fixed device + clocks, pinned container; warmup + ≥30 measured iterations; compare median and p95 against a stored baseline with a noise-calibrated threshold (e.g., fail at >3× observed run-to-run σ); track bytes/FLOPs counters alongside time to catch "slower but also doing more work" confounds; archive traces from failures. (This is Phase 09's accuracy-regression CI with time as the metric — same statistical discipline.)

Q: Your trace shows 40% of step time in aten::copy_. Hypotheses? Host↔device transfers on the hot path (check direction and pageable-vs-pinned), layout conversions from contiguous() after transposes, dtype casts (FP32↔FP16 boundaries), or gradient/optimizer copies in training. Each has a distinct source stack in the trace — follow the stack, not the op name.

References

• Williams et al., Roofline: An Insightful Visual Performance Model for Multicore Architectures (CACM 2009)
• NVIDIA: GPU Performance Background — the math/memory/overhead trichotomy
• Kaplan et al., Scaling Laws appendix (FLOP counting conventions) — arXiv:2001.08361
• torch.profiler docs and PyTorch profiling recipe
• Nsight Compute roofline analysis
• He, Making Deep Learning Go Brrrr From First Principles — horace.io/brrr_intro.html (the best informal treatment of Chapter 5)
• Chen, LLM Inference arithmetic — kipp.ly/transformer-inference-arithmetic (worked KV/bandwidth numbers)