Phase 13 — Fine-Tuning and Adaptation
When and how to change the model's weights — the decision ladder (prompt → few-shot → RAG → fine-tune), the methods (SFT, LoRA/QLoRA, DPO, distillation), the data that makes or breaks them (synthetic generation, dataset quality), and the deployment lifecycle that turns a checkpoint into a maintained production asset.
Why this phase matters
Fine-tuning is the heaviest, last-resort lever — and the most misused. The cardinal rule: fine-tuning teaches behavior, not facts (use RAG for knowledge, Phase 9); reach for it only after prompting → few-shot → RAG fall short (00), because every fine-tune is a standing maintenance commitment. When you do fine-tune, the method is rarely the hard part — data quality dominates (06), and LoRA/QLoRA make it practical on modest hardware (02). This phase covers the full arc: deciding whether to fine-tune, the core methods (imitation via SFT, preference via DPO, compression via distillation), the data (synthetic generation and the quality discipline that prevents collapse/leakage/PII), and deployment (multi-LoRA vs merge, eval-gating, versioning, the never-ending maintenance loop). It composes with serving (Phases 6/7) and is gated by evaluation (Phase 12).
Documents
| # | Document | What you'll be able to do |
|---|---|---|
| 00 | When to Fine-Tune | Apply the ladder (prompt→few-shot→RAG→FT); behavior-not-facts; weigh the maintenance burden |
| 01 | Supervised Fine-Tuning (SFT) | Train on (input → ideal output) pairs; loss masking; overfitting/catastrophic forgetting |
| 02 | LoRA and QLoRA | Train a tiny low-rank adapter on a frozen (4-bit) base; swappable adapters; rank/alpha/targets |
| 03 | DPO and Preference Tuning | Learn from (chosen ≻ rejected); RLHF vs DPO; SFT-first ordering; avoid drift |
| 04 | Distillation | Teach a small student a teacher's behavior; hard vs soft label; quality kept vs cost saved |
| 05 | Synthetic Data | Generate training data; avoid collapse/artifacts; filter, ground, mix; keep eval real |
| 06 | Dataset Quality | Quality > quantity; consistency/diversity/no-contradiction; no leakage; scrub PII |
| 07 | Fine-Tuned Model Deployment | Multi-LoRA vs merge; eval-gate; version/rollback; monitor drift; re-tune/retire |
How to work through it
Start with 00 — most of the value is learning when not to fine-tune (the ladder; behavior-not-facts; the maintenance burden). 01–04 are the methods, in increasing specialization: SFT (imitate ideal outputs) is the foundation; LoRA/QLoRA is how SFT/DPO is done efficiently in practice; DPO refines toward preferred behavior (after SFT); distillation compresses a capable teacher into a cheap student. 05–06 are the data layer that actually determines success — synthetic data (generate to bootstrap/scale/balance, with validation) and dataset quality (the single biggest lever: clean, consistent, diverse, leak-free, PII-free). 07 closes the loop: serving (multi-LoRA vs merge), eval-gating, versioning, and the ongoing maintenance that makes a fine-tune a living asset. Every doc opens with a from-zero plain-English primer and ends with a buildable lab.
Phase 13 artifacts
- A fine-tune-or-not decision using the ladder + a behavior-vs-facts check (00).
- An SFT run on (input → ideal output) pairs with loss masking + overfitting/forgetting checks (01).
- A QLoRA adapter (fits modest VRAM) + a multi-adapter swap demo (02).
- A DPO (LoRA) run on (chosen, rejected) pairs + a before/after + regression check (03).
- A distilled student + a student-vs-teacher eval (quality kept vs cost/latency saved) (04).
- A validated, grounded, mixed synthetic dataset + a raw-vs-validated comparison on a real eval (05).
- A dataset audit + cleaned/deduped/leak-free/PII-scrubbed set + a quality-over-quantity demo (06).
- A served fine-tune (multi-LoRA or merged) that passed an eval gate, with versioning/rollback + a re-tune/retire plan (07).