Lab 01 — Affine / Symmetric / Per-Channel / AWQ Quantization Engine

Phase: 06 — Quantization & Model Compression Difficulty: ⭐⭐⭐☆☆ (the arithmetic is grade-school; the insights are ⭐⭐⭐⭐⭐) Time: 3–4 hours

Quantization is how a 70B model goes from "two 80 GB GPUs" to "one with room to spare," and how decode (memory-bandwidth-bound — Phase 00) gets faster by moving fewer bytes. This lab builds the engine: pick a scale and zero-point that map a real range onto a small integer grid, round-and-clamp onto it, store the codes plus the scale — then the two moves that separate a senior from a tutorial: do it per-channel so a tensor with wildly different per-channel magnitudes is not dragged down by its loudest row, and do activation-aware (AWQ) scaling so the weight columns that multiply large activations keep their precision. Two tests are the soul of the lab: per-channel beats per-tensor, and AWQ beats naive on a salient channel.

What you build

• compute_scale_zeropoint(xmin, xmax, qmin, qmax, symmetric) — the affine map (scale = range/steps, zero_point = round(qmin − xmin/scale)) and the symmetric variant (zero_point == 0, scale from the larger magnitude). The two-line decision that underlies every quantizer.
• quantize_affine / dequantize_affine — round-and-clamp onto [qmin, qmax], and the inverse (q − zp)·scale. The clamp is why outliers saturate instead of wrapping — and the seed of the whole salient-channel problem.
• quantize_tensor(values, n_bits, symmetric, per_channel, axis) + dequantize_tensor — per-tensor (one scale) vs per-channel (one scale per row). Returns codes + scales (+ zero-points) so dequant is exact-as-possible.
• quant_error_mse — the objective every scheme is quietly minimizing.
• apply_awq_scaling / awq_reconstruct / awq_output_error / awq_scale_search — scale salient columns up by act_scale**alpha, quantize, undo the scaling, and pick the alpha that minimizes the activation-weighted output error (‖Wx − Ŵx‖, the error AWQ actually targets).

Key concepts

Files

Run

pip install -r requirements.txt
pytest test_lab.py -v                       # against your lab.py
LAB_MODULE=solution pytest test_lab.py -v   # against the reference
python solution.py                          # the worked example

Success criteria

• All tests pass against your implementation (and LAB_MODULE=solution).
• You can derive scale = (xmax − xmin)/(qmax − qmin) and zero_point = round(qmin − xmin/scale) from "map this range onto that grid," and say why symmetric forces zero_point = 0.
• You can explain why the round-trip error is bounded by scale/2, and why that bound is the same for every value on a uniform grid.
• You can explain why test_per_channel_beats_per_tensor_on_mixed_magnitudes is the soul of the lab — one shared scale set by the loud channel is too coarse for the quiet one.
• You can explain why test_awq_reduces_error_on_salient_channel works: AWQ minimizes the output error, and scaling the salient column up keeps its bits where it matters.
• You can explain why int4 is so much worse than int8 on the same data, and what NF4 / GPTQ / per-group do about it.

How this maps to the real stack

Limits of the miniature (be honest in the interview): we quantize 2-D tensors with a single round, not real fp16 blocks; GPTQ's second-order error compensation (the Hessian / OBQ update that propagates each column's rounding error into the not-yet-quantized columns) is not here — we do data-free min/max calibration, not error-feedback; NF4's non-uniform (normal-float) grid and double-quantization (QLoRA) are described in the WARMUP but not implemented; and real AWQ clips and searches per-group with a proper calibration set, where we use a crafted outlier and a small alpha grid. The mechanisms are faithful; the production polish is the extension list.

Extensions (build these on real hardware)

• Add per-group quantization (one scale per 64/128 weights within a row) and show it sits between per-tensor and per-channel on the error/overhead curve — it's what Q4_K / GPTQ group_size=128 actually use.
• Implement an NF4 grid: 16 non-uniform levels placed on the quantiles of a unit normal, and show it beats uniform int4 on Gaussian weights (the QLoRA result).
• Add GPTQ-style error feedback: quantize columns left-to-right, push each column's rounding error into the remaining columns (the OBQ/Hessian update), and show lower error than min/max.
• Add KV-cache quantization: quantize the K/V tensors from Phase 00's kv_cache_bytes model to int8 and measure the memory win vs the perplexity hit.
• Run the real thing: quantize a HF model with AutoAWQ / auto-gptq / bitsandbytes load_in_4bit, and compare model.get_memory_footprint() and an eval before/after.

Interview / resume

• Talking points: "Walk me through quantizing a weight matrix to int4 — scale, zero-point, clamp." "Why per-channel over per-tensor, and when does per-group win?" "Why are activations harder to quantize than weights, and what do SmoothQuant and AWQ do about it?" "What does GPTQ add over min/max calibration?" "int8 vs int4 vs NF4 — where's the quality cliff?"
• Resume bullet: Built a from-scratch quantization engine — affine and symmetric scale/zero-point calibration, round-and-clamp int8/int4 quantization, per-channel and activation-aware (AWQ) scaling that minimizes activation-weighted output error — demonstrating per-channel and AWQ error reductions on salient-channel tensors.