Lab 03 — PEFT/LoRA Adapters & the Hub: Fine-Tune Small, Ship Versioned

Phase 30 · Lab 03 · Phase README · Warmup

The problem

Full fine-tuning of a 7B-parameter model updates every weight: you need optimizer state for all of them (roughly 4x the model in memory for Adam), and you get a full multi-gigabyte checkpoint per task. LoRA (Low-Rank Adaptation) observes that fine-tuning updates have low intrinsic rank, freezes the base weight matrix W, and learns a factored update instead:

$$ W' = W + \frac{\alpha}{r} , A B \qquad A \in \mathbb{R}^{d_{out} \times r},; B \in \mathbb{R}^{r \times d_{in}},; r \ll \min(d_{out}, d_{in}) $$

Only A and B train — typically well under 1% of the parameters — and at deploy time you choose: merge the product into W (zero inference overhead, the adapter vanishes) or keep it separate and swap adapters per task (one shared base model, many megabyte-sized adapters). The arithmetic identity that makes merging safe — merged forward output equals unmerged forward output, exactly — is something you should be able to prove, and this lab's tests do.

The second half is where fine-tuned artifacts live: the Hub. from_pretrained("org/model", revision=...) is a query against a versioned artifact store — every repo state is a commit, tags name revisions, main floats, and pinning a revision is the difference between a reproducible deployment and "the model changed under us on Tuesday." The Hub is also where the safetensors vs pickle decision bites: pickle deserialization can execute arbitrary code, so a safe loader refuses it unless you explicitly opt in — the trust_remote_code=True supply-chain decision, made mechanical.

What you build

File map

Run it

pytest test_lab.py -v                       # your lab.py (red until you implement)
LAB_MODULE=solution pytest test_lab.py -v   # the reference (green)
python solution.py                          # worked example

Success criteria

• A fresh LoRALinear is an exact no-op (B starts at zeros): forward equals W@x.
• After training, forward(x) matches the hand-computed W@x + (alpha/r)·(A@(B@x)) — the formula, verified numerically.
• trainable_parameters() exposes only lora_A/lora_B, and repeated gradient steps leave base_weight byte-identical.
• merge() changes W by exactly delta_weight() and the merged forward equals the unmerged forward; double-merge and train-after-merge are rejected.
• Loading a different adapter into the same layer changes the output; a rank/shape mismatch raises AdapterError.
• push → from_pretrained round-trips files, card, and sha; commits are full snapshots; shas are deterministic.
• revision="v1.0" keeps returning the pinned commit after new pushes to main.
• A pickle-format artifact is refused without trust_remote_code=True and loads with it; safetensors loads without any flag.
• All 31 tests pass under both lab and solution.

How this maps to the real stack

• peft.LoraConfig(r=8, lora_alpha=16, target_modules=["q_proj","v_proj"]) and get_peft_model(model, config) are the real API: PEFT walks the model, finds the named target modules (attention projections are the classic choice, per the LoRA paper), and wraps each in a lora.Linear that carries frozen base weights plus trainable lora_A/lora_B — structurally our LoRALinear. Real PEFT initializes A with a small random Gaussian and B with zeros; either way the product starts at zero, which is the invariant our zero-init keeps.
• model.merge_and_unload() is our merge(): fold scaling·(A@B) into the base weight and drop the adapter path. Real PEFT also supports unmerge(), multiple named adapters on one model with set_adapter() switching between them — our load_state_dict swap is that mechanism's core.
• QLoRA is this exact machinery with one change: the frozen base weights are stored 4-bit (NF4 quantization via bitsandbytes) while A/B stay in higher precision — which is why a QLoRA adapter is just a LoRA adapter; the quantization lives in the base. The Warmup covers the NF4/double-quantization details.
• huggingface_hub is the real client: push_to_hub/upload_file commit to a git-backed repo, from_pretrained(repo_id, revision=...) accepts a branch, tag, or commit sha, and hf_hub_download caches by revision. Our full-snapshot commits and sha-N counters are the deterministic miniature of the same content-addressed history.
• Model cards really are README.md files with YAML front-matter (license, tags, datasets, metrics) — our ModelCard dataclass is that front-matter. License and gated-access checks are a real pre-adoption step, not paperwork (Warmup §12).
• safetensors vs pickle is a real security boundary: PyTorch .bin checkpoints are pickle and can execute code on load; safetensors is a pure tensor format that cannot. The Hub scans for malicious pickles, transformers defaults to safetensors when available, and trust_remote_code=True (needed for repos that ship custom modeling code) is the explicit opt-in our flag models. Real trust_remote_code gates custom code execution specifically; our miniature folds the pickle risk and the custom-code risk into one gate to keep the lesson (safe-by-default, explicit opt-in) sharp.
• load_adapter returning the sha mirrors the production discipline of recording exactly which artifact revision is serving — the same pin-your-dependencies rule as a container digest.

Limits of the miniature. Real LoRA wraps many modules across many transformer layers and trains with an actual optimizer over batched activations; ours is one linear layer with an injected "gradient step," because the invariants (what may change, what the merge preserves) are the lesson — the training loop itself is Phase 30's Warmup §9 (Trainer) and the transformer internals are the sibling Senior AI Engineer track. The real Hub is git + LFS with content-addressed blobs, resumable downloads, and a global CDN; ours is a dict with counters. Real trust_remote_code executes repo-provided Python classes; nothing here executes anything — the flag models the decision, not the mechanism.

Extensions (your own machine)

• Add unmerge(): subtract delta_weight() back out and prove forward returns to the pre-merge adapter path (floating-point tolerance applies — say why).
• Add multiple named adapters (add_adapter(name) / set_adapter(name)) on one layer, the way PEFT hosts several tasks on one base model.
• Simulate QLoRA storage: quantize the base weights to 4-bit integers with a per-row scale, dequantize in forward, and measure the output error against the float base.
• With real libraries: peft.get_peft_model over a small transformers model, print model.print_trainable_parameters(), and confirm the ratio matches your num_trainable math.

Interview / resume signal

"Implemented LoRA from the formula: a frozen base weight wrapped with rank-r A·B factors scaled by alpha/r, with tests proving the freeze invariant (only A/B ever change), the merge identity (merged forward == unmerged forward, exactly), and adapter save/load/swap — plus a versioned Hub client with full-snapshot commits, tag/sha revision pinning, and safe-by-default loading that refuses pickle artifacts without an explicit trust_remote_code opt-in."