Lab 01 — AWQ Quantization

Phase: 10 — GenAI on Edge | Difficulty: ⭐⭐⭐⭐⭐ | Time: 3–4 hours

AWQ (Activation-aware Weight Quantization, Lin et al., 2023) is state-of-the-art INT4 quantization for LLMs deployed on edge devices.

What you build

  • compute_per_channel_scale — max|W[i,:]| / 7 for INT4 symmetric quantization
  • quantize_weight_int4 — clamp[-7,7] → round → dequantize
  • awq_scale_search — grid search over α∈[0,1]: s = x_max^α, minimize ‖W@X - Q(W·s)/s@X‖²
  • awq_quantize_linear — full AWQ quantization of a single linear layer
  • evaluate_quantization_error — relative error + cosine similarity

Key concepts

ConceptWhat to understand
Channel scalingMultiply salient weight channels by s before quantization (reduces error)
Activation-awares = x_max^α — scale determined by activation magnitude, not just weights
INT4 vs INT8INT4 = 2× more layers fit in NPU SRAM — critical for 7B+ edge LLMs
AWQ vs GPTQAWQ: activation scaling; GPTQ: Hessian-based correction — both SOTA

Files

FilePurpose
lab.pyStubs with TODO markers
solution.pyComplete working implementation
test_lab.pypytest suite
requirements.txtDependencies

Run

pip install -r requirements.txt
python lab.py
pytest test_lab.py -v