Lab 01 — Functional Data SDK (Validated, Resource, IO)

Phase: 12 — Scala & Functional Data Engineering | Difficulty: ⭐⭐⭐⭐⭐ | Time: 6–8 hours

The JD's Round 4 is "design a Scala library with Cats abstractions, resource safety, effect systems, error modeling, and retry semantics." This lab makes you build those abstractions so you understand them from the inside — Validated (accumulating), Result (fail-fast monad), Resource/bracket (guaranteed LIFO release), IO (lazy effect / referential transparency), and retry (an effect combinator).

Two files, one set of ideas

  • lab.py / solution.py / test_lab.py — a Python companion that encodes the exact semantics so the tests run anywhere (no JVM/sbt needed). This is what you implement.
  • reference.scala — the authentic Cats / Cats-Effect / FS2 version an interviewer expects: ValidatedNel + mapN, EitherT, Resource.make, IO, FS2 Stream with parEvalMap backpressure, retry with exponential backoff. Read it; run it with sbt if you have a JVM. The Python tests verify the properties; the Scala is for fluency.

What you build (in Python)

AbstractionCats/ZIO equivalentProperty the test pins
Validated.map2Validated / mapNaccumulates all errors
Result.flat_mapEither / EitherTfail-fast (stops at first error)
bracket / ResourceResource / bracketrelease even on exception, LIFO
IOcats.effect.IO / ZIOlazy: describe ≠ run; runs each time
retryretry combinatorbounded attempts, still lazy

Run

pip install -r requirements.txt
pytest test_lab.py -v
LAB_MODULE=solution pytest test_lab.py -v

Success criteria

  • All 19 tests pass — test_validated_map2_accumulates_errors (applicative vs monad), test_resource_releases_lifo_on_exception (resource safety), and test_io_is_lazy_until_run (referential transparency) are the ones that certify understanding.
  • You can explain why Validated is an applicative (accumulates) and Result/Either is a monad (fail-fast), and when each is correct.
  • You can explain why a description-of-an-effect (IO) that does nothing until run() is more composable and testable than eager side effects.

Extensions

  • Add a Ref-style atomic mutable cell and a tiny parMapN to model concurrency (Cats Effect Ref / parMapN).
  • Add an FS2-style lazy Stream (pull-based) with take, map, evalMap, and a bounded parEvalMap — and show resource safety across the stream.
  • Port one function to real Scala in reference.scala and run it with sbt.
  • Build a typed EventContract[A] that combines a schema (P03) + Validated decoding into a reusable SDK type — the seed of the platform library the JD asks for.