Lab 02 — Schema Compatibility & Data Contracts in Scala

Phase: 03 — Serialization, Schema & Data Contracts | Difficulty: ⭐⭐⭐⭐☆ | Time: 4–5 hours

The Scala twin of Lab 01's Python registry — the schema-governance logic a JVM platform team ships as a library. Compatibility checking (backward/forward/full + Protobuf field-number rules) plus Cats Validated record validation that accumulates every violation in one pass. Verified with ScalaTest via sbt test.

What you build

  • Schema / Field — a schema model with Protobuf field numbers and the hasDefault rule that makes add/remove safety mechanical
  • Compatibility — backwardViolations, forwardViolations, compatibilityViolations (none/backward/forward/full), and protobufViolations (reused tag + changed type = silent byte corruption)
  • Validation.validateRecord — Cats ValidatedNel validation accumulating missing-required, type-mismatch, and semantic-rule failures (a producer fixes them all at once)

Run

sbt test

First run downloads Scala 2.13 + Cats + ScalaTest, then compiles and runs. Expected: 11 tests, all green across CompatibilitySpec and ValidationSpec.

Key concepts (Scala expression)

ConceptScala
Backward compatnew reader, old data → added field needs a default
Forward compatold reader, new data → removed field needed a default
Protobuf tagsfield NUMBER is the wire identity; reuse+retype corrupts bytes
Accumulating validationValidated mapN/fold collects all errors (vs Either fail-fast)

Success criteria

  • sbt test → all 11 specs pass.
  • You can explain why a renamed field with the same number+type is Protobuf-safe but a retyped reused number is catastrophic.
  • You can explain why validateRecord uses Validated (accumulate) not Either (fail-fast).

Extensions

  • Add enum evolution rules (adding a symbol is backward-safe; removing isn't).
  • Add a CI gate function: assertCompatible(old, new, mode) that throws with all violations — the pre-merge check (P03 WARMUP Ch. 7).
  • Replace typ: String with a sealed FieldType ADT for exhaustive, typo-proof matching (P12's make-illegal-states-unrepresentable).