Lab 01 — Typed Data-Contract Library

Phase: 01 — Production Python Engineering | Difficulty: ⭐⭐⭐☆☆ | Time: 4–6 hours

Every production ML pipeline has boundaries where data arrives from systems you don't control. This lab builds the boundary guard — a miniature pydantic/pandera — and in doing so drills the production-Python toolkit: protocols, frozen dataclasses, collect-don't-raise error design, and property-style testing.

What you build

  • ValidationIssue / ValidationReport — frozen value objects; issues carry path + rule + message so a million-record batch aggregates into a dashboard
  • Built-in rules — TypeRule (with the bool-is-not-int trap), RangeRule, RegexRule (fullmatch!), EnumRule — each a frozen dataclass satisfying a structural Rule protocol
  • Schema.validate — collects every issue: unknown fields, missing required, nullability, coercion failures, rule failures
  • Coercion — "42" → 42 where declared, failures as issues, never exceptions
  • Composition — extend / subset returning new schemas (immutability pays off)

Key concepts

ConceptWhat to understand
Collect-don't-raiseData problems are values (issues), not exceptions; exceptions are for programmer errors (see extend's clash check)
Structural typingA custom rule is any object with name + check() — the test proves it with a class that inherits nothing
bool ⊂ intPython's subclassing trap: a data contract that accepts True as an int corrupts downstream aggregates
fullmatch vs searchsearch would accept "xu_12y" for pattern u_\d+ — boundary anchoring is contract semantics
Frozen compositionextend/subset return new schemas because the originals can't change — share schemas fearlessly

Files

FilePurpose
lab.pySkeleton with # TODO markers
solution.pyComplete reference; python solution.py runs a demo
test_lab.py15 tests incl. the property-style single-mutation localization test
requirements.txtpytest only — pure stdlib library

Run

pytest test_lab.py -v                       # your lab.py
LAB_MODULE=solution pytest test_lab.py -v   # the reference

Success criteria

  • All 15 tests pass — especially test_property_single_field_mutation_localizes_issue (mutating exactly one field must produce issues at exactly that path)
  • You can explain why validate returns a report while extend raises — the data-error vs programmer-error line, drawn correctly
  • You can write a new rule (e.g., LengthRule) in under two minutes without touching the library

Extensions

  • Nested schemas (Field(dtype=Schema)) with dotted paths (user.address.zip)
  • A pandas adapter: validate a DataFrame column-wise, returning issue counts per rule (the data-quality dashboard primitive Phase 02 uses)
  • Compare your design to pydantic v2's — what does compiled-core validation buy, and what did you not need?

References

  • pydantic v2 docs and pandera — the industrial versions
  • Alexis King, Parse, don't validate
  • WARMUP.md chapters 2–4 — the design rationale in full