Dataset Quality

Phase 13 · Document 06 · Fine-Tuning and Adaptation Prev: 05 — Synthetic Data · Up: Phase 13 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

Dataset quality is the single biggest lever in fine-tuning — far more than the method (SFT/LoRA/DPO), the hyperparameters, or the base model. The model imitates exactly what you show it (01): contradictory examples teach contradictions, sloppy formatting teaches sloppiness, biased data teaches bias, and a leaked test example inflates your eval into a lie. The hard-won industry lesson is a few hundred to a few thousand clean, consistent, diverse examples beat tens of thousands of noisy ones. This doc is the discipline that makes every other doc in Phase 13 actually work: how to build, clean, balance, and protect a fine-tuning dataset — and the failure modes (contradictions, leakage, PII, low diversity) that quietly wreck fine-tunes.

2. Core Concept

Plain-English primer: the model becomes your dataset

A fine-tuned model is a mirror of its training data (01). It doesn't learn what you meant — it learns the patterns actually present in the examples. So every property of your dataset becomes a property of your model:

• Show consistent format/style → it learns that format. Show inconsistent examples → it learns to be inconsistent (or picks the majority and ignores your intent).
• Show contradictions (same input, different "ideal" outputs) → it learns confusion / averages them.
• Show biased/toxic examples → it amplifies them.
• Show errors → it learns the errors as if correct.

This is why dataset quality dominates: the data IS the spec. Garbage in, garbage out — literally.

Quality beats quantity

The most important practical fact: clean, consistent, diverse data in modest amounts beats large noisy datasets. Famous results (e.g., LIMA — "Less Is More for Alignment") showed ~1,000 carefully curated examples can rival much larger sets. The reason: the base model already has the capability (00); fine-tuning just surfaces/shapes it, and a small set of crisp examples does that cleanly, while noise actively teaches the wrong thing.

RIGHT: a few hundred–few thousand CLEAN, CONSISTENT, DIVERSE examples  → crisp, reliable behavior
WRONG: tens of thousands of NOISY/contradictory/duplicated examples    → confused, inconsistent model

The dimensions of dataset quality

1. Correctness — every output is actually right (and in the desired style). Wrong examples teach wrong behavior. Verify against ground truth / tests / review (05).
2. Consistency — same format, style, structure, and decision logic across all examples. Inconsistency is the most common silent killer (10 labelers → 10 styles). Use a style guide / schema and reconcile disagreements.
3. No contradictions — don't have near-identical inputs mapped to conflicting outputs; the model can't learn a coherent rule from contradictions.
4. Diversity & coverage — span the real input distribution: varied phrasings, lengths, topics, and edge cases / hard negatives. A narrow dataset overfits to a slice and fails on the rest. (Hard negatives — tricky cases near the decision boundary — are especially valuable, 05.)
5. Balance — don't over-represent easy/common cases and starve rare/important ones; balance classes and difficulty so the model doesn't just learn the majority.
6. Right difficulty / informativeness — examples should teach something; trivial or redundant examples add cost without signal. Deduplicate.
7. Clean formatting — exact, consistent structure (chat template, delimiters, JSON schema) — the model learns formatting literally (01).

The non-negotiables: leakage and PII

• No train/eval leakage — your eval/golden set must be disjoint from training data (Phase 12.01). If test examples (or near-duplicates) leak into training, your eval measures memorization, not generalization — inflated scores that don't transfer. Deduplicate across the split, not just within.
• PII / sensitive data — training data gets baked into the weights and can be regurgitated. Scrub or pseudonymize PII, secrets, credentials (Phase 14.03); respect consent/licensing. A model that memorizes a customer's SSN and emits it later is a serious incident.
• Bias & safety — audit for harmful/biased content; it gets amplified. This is part of the dataset, not an afterthought.

Building the dataset (the workflow)

1. Define the spec — exact task, format, style; write a labeling/style guide so examples are consistent.
2. Source — real production data (best, Phase 12.01), human labeling, or synthetic (validated, 05).
3. Clean — dedup (within and across the eval split), fix/verify correctness, normalize formatting, scrub PII.
4. Balance & diversify — fill coverage gaps and edge cases (often with targeted synthetic data, 05).
5. Reconcile — resolve contradictions and inter-labeler inconsistencies against the spec.
6. Split — hold out a clean, real, disjoint eval set (Phase 12.01); never train on it.
7. Iterate from errors — inspect model mistakes, add targeted examples (data-centric loop), re-train, re-eval.

Size guidance (rough, task-dependent)

• Hundreds of clean examples can move format/style/narrow behavior (esp. with LoRA, 02).
• Low thousands for more complex behaviors.
• More only helps if it stays clean and diverse — adding noise to scale up backfires. Quality first, then scale.

3. Mental Model

   THE MODEL IS A MIRROR OF ITS DATA [01]: it imitates the patterns ACTUALLY present, not what you meant. DATA = THE SPEC.
   ★ QUALITY > QUANTITY: a few hundred–few thousand CLEAN/CONSISTENT/DIVERSE > tens of thousands NOISY (LIMA: less is more)

   DIMENSIONS: correctness · CONSISTENCY (style guide/schema) · NO contradictions · DIVERSITY+coverage (edge cases/hard negatives [05]) ·
               balance · informativeness (dedup) · clean formatting (learned literally [01])
   NON-NEGOTIABLES: NO train/eval LEAKAGE (disjoint, dedup across split → else you measure memorization [12.01]) ·
                    scrub PII/secrets (baked into weights, regurgitated [14]) · audit bias/safety

   WORKFLOW: spec/style guide → source (real [12.01] / label / validated synthetic [05]) → clean (dedup, verify, scrub PII) →
             balance+diversify [05] → reconcile contradictions → SPLIT clean disjoint eval [12.01] → iterate FROM ERRORS
   SIZE: hundreds (format/style w/ LoRA [02]) · low-thousands (complex); more helps ONLY if it stays clean+diverse

Mnemonic: the model becomes its dataset — it learns the patterns you actually show it. Quality beats quantity: a few hundred clean, consistent, diverse examples beat tens of thousands of noisy ones. Never leak eval into train; never bake in PII.

4. Hitchhiker's Guide

What to look for first: are the examples correct, consistent, and diverse, and is your eval set disjoint from training? Those three (correctness, consistency, no-leakage) decide whether the fine-tune means anything.

What to ignore at first: chasing dataset size. Get clean and consistent at small scale first; scale only while quality holds.

What misleads beginners:

• "More data = better." Noisy scale backfires — quality beats quantity (LIMA).
• Inconsistent examples. Multiple labelers/styles teach inconsistency — use a style guide/schema and reconcile.
• Contradictions. Same input → conflicting outputs can't form a rule; the model averages/confuses.
• Train/eval leakage. Test examples (or near-dupes) in training inflate eval into memorization — dedup across the split (Phase 12.01).
• Ignoring PII. Training data is memorized and regurgitated — scrub PII/secrets (Phase 14).
• Narrow coverage. Missing edge cases/hard negatives → fails outside the slice (05).

How experts reason: they treat the dataset as the spec — write a style guide, source from real data where possible (Phase 12.01), ruthlessly clean (dedup, verify correctness, scrub PII), balance and diversify (targeted synthetic for edge cases, 05), reconcile contradictions, hold out a clean disjoint eval, and iterate from the model's actual errors (data-centric loop). They keep it small and clean before scaling.

What matters in production: consistency/correctness of the data, no leakage into eval, no PII in weights, coverage of real inputs/edge cases, and a clean held-out eval to trust (Phase 12).

How to debug/verify: inconsistent model outputs → inconsistent training data; model fails on edge cases → coverage gap (add hard negatives); great eval, bad prod → leakage (Phase 12.01); model emits PII → unscrubbed training data; behavior averages two styles → contradictions.

Questions to ask: is every example correct and consistent? are there contradictions? does it cover the real distribution + edge cases? is the eval set disjoint (deduped across split)? is PII scrubbed? am I scaling clean data or just adding noise?

What silently gets expensive/unreliable: inconsistency (the #1 silent killer), train/eval leakage (lying metrics), PII regurgitation (incident), narrow coverage (prod failures), and scaling noise instead of quality.

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• The fine-tuned model is a mirror of its dataset — it imitates the patterns actually present, not your intent; the data is the spec (01).
• Quality beats quantity — a few hundred to a few thousand clean, consistent, diverse examples beat tens of thousands of noisy ones (LIMA).
• Dimensions: correctness, consistency, no contradictions, diversity/coverage (edge cases/hard negatives), balance, informativeness (dedup), clean formatting.
• Consistency is the #1 silent killer — use a style guide/schema; reconcile inter-labeler disagreement.
• No train/eval leakage — the eval set must be disjoint (dedup across the split) or you measure memorization (Phase 12.01).
• PII/secrets get baked into weights and regurgitated — scrub/pseudonymize; respect consent/licensing (Phase 14).
• Audit for bias/safety — biased or toxic data gets amplified.
• Iterate from the model's errors (data-centric loop): inspect mistakes → add targeted examples → re-train → re-eval. Quality first, then scale.

9. Observations from Real Systems

• LIMA (~1,000 curated examples) rivaled far larger instruction sets — the canonical evidence for quality over quantity.
• Data-centric AI (Andrew Ng) reframed ML progress around improving data rather than models — the dominant practical mindset for fine-tuning.
• Memorization research shows models regurgitate training data verbatim — why PII/secret scrubbing is non-negotiable (Phase 14).
• Cross-split deduplication routinely catches leakage that inflates benchmarks; teams dedup train against eval, not just within train (Phase 12.01).
• The most common real fine-tune failure is inconsistent labels — different annotators/styles producing a model that's confused or averages behaviors.

10. Common Misconceptions

11. Engineering Decision Framework

BUILD A FINE-TUNING DATASET (data = the spec [01]):
 1. SPEC: define task/format/style; write a labeling/style guide (consistency).
 2. SOURCE: real production data (best [12.01]) / human labels / VALIDATED synthetic [05].
 3. CLEAN: dedup (within AND across eval split), verify correctness, normalize formatting, SCRUB PII/secrets [14].
 4. BALANCE + DIVERSIFY: cover the real distribution + edge cases / hard negatives (targeted synthetic [05]).
 5. RECONCILE: resolve contradictions and inter-labeler inconsistency against the spec.
 6. SPLIT: hold out a CLEAN, REAL, DISJOINT eval set [12.01] — never train on it.
 7. ITERATE FROM ERRORS: inspect mistakes → add targeted examples → re-train → re-eval. Quality first, THEN scale.

12. Hands-On Lab

Goal

Take a raw example set, audit and clean it (consistency, dedup, leakage, PII, coverage), and show the cleaned dataset trains a better model than the raw one — on a clean held-out eval.

Prerequisites

• A raw (input → output) set (real or synthetic, 05); the SFT/LoRA stack (01/02); a real, held-out eval set (Phase 12.01).

Steps

1. Audit: measure consistency (format/style variance), find contradictions (same input, different output), duplicates, coverage gaps/edge cases, and scan for PII.
2. Check leakage: dedup training against the eval set (exact + near-duplicate); remove any overlap (Phase 12.01).
3. Clean: normalize formatting to one schema/style, fix/verify correctness, reconcile contradictions, scrub PII.
4. Diversify: add edge cases / hard negatives (targeted synthetic, 05) to fill coverage gaps.
5. Train both: SFT (LoRA) on raw vs cleaned datasets (01/02).
6. Eval both on the clean held-out set (Phase 12.01); compare — cleaned should win, often with fewer examples.

Expected output

A cleaned dataset (more consistent, deduped, leak-free, PII-scrubbed, better coverage) that beats the raw set on a real eval — often with fewer examples — proving quality over quantity and the cost of leakage/inconsistency.

Debugging tips

• Cleaned didn't win → check you actually removed contradictions/leakage; verify the eval set is truly disjoint.
• Suspiciously high raw score → leakage; dedup train↔eval (Phase 12.01).

Extension task

Quantify consistency (style variance) and diversity (n-gram/embedding spread) before/after; ablate dataset size (hundreds vs thousands) at equal quality.

Production extension

Stand up a data-centric loop: capture production errors → add targeted, consistent examples → re-train → eval-gate (Phase 12.08); enforce PII scrubbing and cross-split dedup in the pipeline (Phase 14).

What to measure

Consistency/diversity before-after, dedup/leakage removed, PII found/scrubbed, raw-vs-cleaned quality on real eval, examples-vs-quality.

Deliverables

• An audit report (consistency, contradictions, dupes, leakage, PII, coverage).
• A cleaned, deduped, leak-free, PII-scrubbed dataset with added edge cases.
• A raw-vs-cleaned training comparison on a real eval (quality > quantity demonstrated).

13. Verification Questions

Basic

1. Why is the dataset "the spec" for a fine-tuned model?
2. Why does quality beat quantity (cite the intuition)?
3. What is train/eval leakage and why does it inflate metrics?

Applied 4. Name five dimensions of dataset quality. 5. Why must PII be scrubbed from training data?

Debugging 6. Your fine-tune gives inconsistent outputs. Likely cause and fix. 7. Eval scores are great but production is poor. What do you check first?

System design 8. Design a data-centric loop that improves a fine-tune from production errors without introducing leakage.

Startup / product 9. With limited labeling budget, how do you get the most from a small dataset, and what are the non-negotiables?

14. Takeaways

1. The fine-tuned model mirrors its dataset — it learns the patterns you actually show it; the data is the spec (01).
2. Quality beats quantity — clean, consistent, diverse examples in modest amounts beat large noisy sets (LIMA).
3. Consistency, no-contradictions, and coverage (edge cases/hard negatives) are the core dials; dedup for informativeness (05).
4. Non-negotiables: no train/eval leakage (disjoint, dedup across split, Phase 12.01) and no PII in weights (Phase 14).
5. Iterate from errors (data-centric loop); quality first, then scale — adding noise backfires.

15. Artifact Checklist

• A dataset audit (consistency, contradictions, dupes, leakage, PII, coverage).
• A cleaned, deduped, leak-free, PII-scrubbed dataset.
• Added edge cases / hard negatives for coverage.
• A clean, disjoint, real held-out eval split.
• A raw-vs-cleaned training comparison (quality > quantity).

Up: Phase 13 Index · Next: 07 — Fine-Tuned Model Deployment