Phase 18 — C++/CUDA & GPU Performance Engineering

The phase where the roofline from Phase 00 stops being a slide and becomes a kernel. Every earlier phase reasoned about a model as a physical system; this one zooms all the way in to the hardware that runs it — the warp, the SM, the memory hierarchy, the kernel — and asks the question that separates a Senior AI Engineer from someone who uses GPUs: given a slow kernel, can you say which physical resource is the bottleneck and what to change, before you ever open a profiler? That is a whiteboard skill, and it is arithmetic.

Why this phase exists

By now you can size a model (P00), build attention (P02), quantize it (P06), and serve it (P09). But the actual speed of all of that is decided one kernel at a time on the GPU, and the people who own that layer — who write the custom op, fuse the attention, fix the coalescing bug, and read the Nsight roofline — are the ones teams cannot replace. You do not need a GPU to think like them. Phase 00 gave you min(peak, bw·I) for a whole model; here we apply the same physics to a single kernel and add the four levers that actually move kernel performance:

1. The execution model — thread → warp (32, SIMT, lockstep) → block → grid, on SMs. Divergence serializes a warp; this is why branchy code is slow.
2. The memory hierarchy — registers → shared memory/L1 → L2 → HBM, each ~an order of magnitude slower and bigger than the last. Where your data lives is the whole game.
3. Occupancy & latency hiding — GPUs hide memory stalls by oversubscribing warps; how many fit is capped by registers, shared memory, or warp/block slots, and which one is the lever.
4. Coalescing — a warp's 32 loads must be contiguous to pack into the fewest memory transactions. This is the single biggest memory-perf lever, and the easiest to get wrong.
5. Tiling & fusion — reuse data on-chip (tiled GEMM) and keep intermediates on-chip (fused kernels, FlashAttention) so you pay HBM bandwidth as rarely as possible.

Get fluent here and you can read any kernel's performance like a balance sheet — and you finally own the bottom of the stack the whole curriculum sits on.

What to revisit first

This phase is the direct descendant of two earlier ones — skim them before you start:

• Phase 00 (Foundations) — the roofline (min(peak, bw·I), ridge = peak/bw) and arithmetic intensity. P18 takes that model-level tool and aims it at a single kernel, then adds occupancy, coalescing, tiling, and fusion. If "decode is I≈1, memory-bound" doesn't feel automatic yet, re-read P00's Chapter 5 first.
• Phase 09 (Inference serving) — continuous batching, PagedAttention, the KV-cache. Half of why those exist is the kernel physics you'll prove here (decode is bandwidth-bound; launch overhead bites tiny kernels). P18 is the why under P09's what.

No GPU is required for the lab. The real CUDA appears only in the lab README's Extensions, for when you have hardware.

Concept map

                 ┌───────────────────────────────────────────┐
                 │  A kernel is a physical system too          │
                 │  (deepen Phase 00's roofline to the metal)  │
                 └───────────────────────────────────────────┘
                    │                │                  │
        ┌───────────┘         ┌──────┘           ┌──────┘
        ▼                     ▼                   ▼
   EXECUTION MODEL       MEMORY HIERARCHY     THE ROOFLINE (per kernel)
   thread→warp(32)       reg→smem/L1→L2→HBM   min(peak, bw·I)
   →block→grid; SMs      fast/small ⟶ slow/big   ridge = peak/bw
   SIMT lockstep         latency hidden by...      I≈1 decode  → mem-bound
   divergence serializes      OCCUPANCY            I≈500 GEMM  → compute-bound
        │                 active_warps/max              │
        │            limiter: reg | smem | warp | block │
        └───────────┬──────────────┬───────────────────┘
                    ▼              ▼
            THE FOUR LEVERS   ┌─────────────────────────────┐
            ─────────────────│ COALESCING  contiguous=1.0   │
            (move fewer       │ TILING      reuse ∝ 1/tile   │
             bytes; keep      │ FUSION      n_ops× less HBM  │
             data on-chip)    │ (mixed precision/tensorcore) │
                              └─────────────────────────────┘
                    │
                    ▼
          C++/CUDA custom op · Triton · torch.compile
          profile: Nsight Compute roofline ≠ nvidia-smi util

The lab

The lab is a runnable, test-verified miniature — see the lab standard. It ships no CUDA: you model the kernel with arithmetic, because reasoning about a kernel without hardware is exactly the senior skill. Run it red (pytest test_lab.py), make it green, read solution.py, then build the real CUDA kernels in the README's Extensions on a GPU.

Integrated scenario ideas

• Triage a slow kernel. Given a kernel's FLOPs, bytes, registers, and shared-memory use: place it on the roofline, compute its occupancy and limiter, estimate its coalescing efficiency, and name the one change with the biggest payoff. (This is a real on-call drill.)
• Defend "this is memory-bound, more FLOPs won't help." Take decode (I≈1), put it on the A100 roofline (ridge 156), and show the fix is batching/quantizing/fusing, not a faster part — tying P18 straight back to P00 and P09.
• Justify the tile size. Show tiling cuts HBM traffic ∝ 1/tile, then show that pushing the tile further blows the register/shared-memory budget and collapses occupancy — the occupancy-vs-reuse tradeoff, with numbers.
• Build-vs-Triton-vs-fuse. Decide whether a hot op warrants a hand-written CUDA C++ extension, a Triton kernel, or just torch.compile — and price each in engineering time vs speedup.
• Explain FlashAttention to a new hire as "fusion + tiling so attention never writes the N×N matrix to HBM" — and show the traffic ratio that makes it a win.

Deliverables checklist

• lab.py passes pytest test_lab.py -v (and LAB_MODULE=solution does too).
• You can draw the execution hierarchy (thread → warp → block → grid → SM → GPU) and explain why a warp is 32 lanes in lockstep and what divergence costs.
• You can list the memory hierarchy (registers → shared/L1 → L2 → HBM) with the rough latency/bandwidth ordering and say which optimization targets which level.
• You can compute an occupancy by hand and name the binding resource.
• You can state why coalescing is the #1 memory lever and estimate efficiency for a stride.
• You can explain tiling and fusion as "move fewer bytes / keep data on-chip" and quantify both.
• You can turn any "this kernel is slow" prompt into: bound → limiter → coalescing → fix.

Key takeaways

• A kernel is a physical system, same as a model. The roofline you learned in P00 applies unchanged to a single kernel: min(peak, bw·I), ridge = peak/bw. The whole job is moving a kernel rightward (more reuse, fewer bytes) or upward (better units, mixed precision).
• Occupancy is necessary-ish, and the limiter is the lever. Knowing you're at 25% is useless; knowing you're register-limited tells you to cut registers (or accept it for the ILP). Always name the binding resource.
• Coalescing is the #1 memory lever and the easiest bug. Contiguous warp accesses are free; strided ones bleed bandwidth proportional to the stride. Most "mysteriously slow" memory kernels are an uncoalesced access (often an array-of-structs that wants to be a struct-of-arrays).
• Tiling and fusion both win the same way: pay HBM less often. Tiling reuses loaded data on chip (traffic ∝ 1/tile); fusion keeps intermediates on chip (n_ops× less traffic). FlashAttention is the canonical fused, IO-aware kernel — and Triton/torch.compile automate the same idea.
• nvidia-smi 100% ≠ efficient — the roofline tells the truth. Utilization means "a kernel ran," not "the math units were busy." A memory-bound decode shows 100% util at single-digit MFU. This is the P00 lesson, now provable at the kernel level with Nsight.
• C++/CUDA is the floor you can always drop to. Most of the time torch.compile/Triton is enough; when it isn't, a hand-written CUDA op behind a PyTorch C++ extension is the escape hatch — and being the person who can write it is the career moat this phase builds.

Next: Phase 19 — Capstone: Production Multimodal Agentic Serving Platform.