Lab 02 — Hardware-Aware Quantization

Phase: 06 — Qualcomm NPU Hardware & Deployment | Difficulty: ⭐⭐⭐⭐⭐ | Time: 3–4 hours

NPU quantization requires hardware-specific constraints: INT8 weights, INT8 activations, symmetric quantization, and specific range policies.

What you build

  • Hardware-constrained quantizer: symmetric INT8, power-of-2 scales
  • Per-layer sensitivity analysis (which layers tolerate INT4 vs need INT8)
  • Mixed-precision assignment policy based on accuracy drop thresholds
  • Validate quantized model against NPU op constraints

Key concepts

ConceptWhat to understand
Symmetric quantNPU typically requires symmetric (zero_point=0) — simpler HW
Power-of-2 scalesSome NPUs require scales = 2^k for efficient shifting
Per-layer sensitivityMeasure ‖W_q @ X - W @ X‖ per layer to find sensitive layers
Mixed precisionINT4 for tolerant layers, INT8 for sensitive — maximize compression

Files

FilePurpose
lab.pyStubs with TODO markers
solution.pyComplete working implementation
test_lab.pypytest suite
requirements.txtDependencies

Run

pip install -r requirements.txt
python lab.py
pytest test_lab.py -v