Lab 01 — Tradeoff & Lifecycle Modeler

Phase: 00 — The Principal Data Infrastructure Engineer | Difficulty: ⭐⭐☆☆☆ | Time: 3–4 hours

Round-1 of a data-platform interview is back-of-envelope arithmetic worn as architecture. This lab builds the calculator: latency budgets, cost models, sizing, quorum math, delivery-semantics classification, and a weighted tradeoff resolver whose whole point is that the same two architectures flip winners when the workload's weights change — which is why "best architecture" is a meaningless phrase.

What you build

  • LifecycleStage — the 11-stage spine of the entire track, as an enum
  • allocate_latency_budget / budget_headroom_ms — decompose an SLA across stages with budget conservation, and find the slack that's a cost lever
  • athena_scan_cost_usd / s3_storage_cost_usd / spot_compute_cost_usd — the three cost models that actually decide architectures
  • kafka_partitions_needed / kinesis_shards_needed / quorum_is_strong — the sizing and consistency arithmetic (R + W > N, and the off-by-one that isn't strong)
  • classify_delivery_semantics — at-most / at-least / exactly-once-effect
  • resolve_tradeoff — score two architectures on the five forces under workload weights, and return the deciding force (the one sentence for your ADR)

Key concepts

ConceptWhat to understand
Budget conservationstage budgets must sum to the SLA; you can't allocate 110%
Spend the slackunused latency is a cost lever, not a virtue
Layout is a cost feature$5/TB makes partitioning/columnar a bill decision
R + W > Nstrict inequality; equality is not strong consistency
Exactly-once-effectthe only exactly-once there is; a per-product dial, not a default
Explainable tradeoffsthe deciding force is what you defend in the review

Files

FilePurpose
lab.pyskeleton with # TODO markers
solution.pycomplete reference; python solution.py runs a worked example
test_lab.pythe proof — run it red, make it green
requirements.txtpytest only (pure stdlib otherwise)

Run

pip install -r requirements.txt
pytest test_lab.py -v                      # against your lab.py
LAB_MODULE=solution pytest test_lab.py -v  # against the reference

Success criteria

  • All tests pass against your implementation.
  • You can explain why test_quorum_strong_boundary asserts quorum_is_strong(3,2,1) is False (3 = 3, not > 3).
  • You can explain why test_weights_flip_the_winner is the whole point of the lab.
  • You can take any "design X" prompt and, in 2 minutes, produce a latency budget and a scan-cost estimate on paper.

Extensions

  • Add gdpr_deletion_cost(...) — model the cost of point-deletes across a partitioned lake (foreshadows P09 time-travel-vs-deletion and P14 retention).
  • Add a recommend_engine(requirements) that returns Kafka-vs-Kinesis-vs-Pulsar (or Flink-vs-Spark-SS) with the deciding force — then defend it against a friend.
  • Make resolve_tradeoff accept N architectures and return a ranked list with pairwise deciding forces.