Interview Prep — The Principal Data Infrastructure Loop

This track maps directly onto the JD's own five-round evaluation loop. Each round below links to the phases that prepare it, lists what interviewers actually probe, and gives the "signal" that separates a senior answer from a principal one. The meta-skill across all five (P00): turn vague prompts into numbers, name the dial, and defend the seam.

Use the per-phase WARMUP.md "Interview Q&A" sections as your question bank, and the CHEATSHEET.md for the numbers. This page is the strategy.

Round 1 — Distributed Data Systems Design

Prompt (JD): Design a real-time platform ingesting 5M events/sec, validating Protobuf, processing with Flink, storing raw + curated in S3/Iceberg, serving Athena queries and Cassandra features.

Prepares it: P00 (tradeoffs), P01 (distributed systems), P02 (Kafka/Kinesis), P03 (Protobuf/contracts), P04 (Flink), P08–P09 (storage/lakehouse), P10 (query), P11 (serving) + system-design/01.

What they probe: Kafka vs Kinesis (with numbers); partitioning & hot keys; schema registry & DLQ; Flink topology + checkpointing + exactly-once sink; storage layout (partition/sort/file size); Cassandra modeling; replay; observability; failure recovery; the cost model.

Senior → Principal signal: A senior names the components. A principal extracts the numbers first (read:write ratio, p99 + percentile, correctness-per-stream, budget, who operates it), decomposes a latency budget (P00), names the dials as constraints are added ("…and cheap" → which dial), and defends the seams (exactly-once effect re-earned at the Cassandra/Iceberg boundary). Say "5M/sec alone tells me little; the ratios and SLAs tell me everything."

Round 2 — Deep Debugging (Streaming)

Prompt (JD): A Flink job has rising checkpoint duration, increasing RocksDB state, delayed watermarks, and growing Kafka lag. Diagnose.

Prepares it: P04 (Flink internals), P01 (backpressure), P13 (observability).

What they probe: Do you see one story or four bugs? Backpressure → barrier/alignment stall → checkpoint duration ↑ + source throttle → lag ↑; state growth from open windows / no TTL / unbounded keys; watermark delay from idle partitions or backpressure.

Senior → Principal signal: Draw the causal chain (P04 Ch. 8) calmly: find the backpressured operator, ask why it's slow (slow sink? hot-key skew? expensive process?), and give the cause-matched fix (async/scale sink, re-key/salt, incremental aggregation, state TTL, savepoint rescale, unaligned checkpoints to relieve alignment). The signal is treating the four symptoms as one root cause.

Round 3 — Spark & Lakehouse Optimization (Batch)

Prompt (JD): A Spark job on EMR runs 9 hours, spills heavily, writes millions of small Parquet files, and makes Athena scan too much. Diagnose.

Prepares it: P06 (Spark internals), P07 (EMR), P08 (Parquet/files), P09 (compaction), P10 (Athena cost).

What they probe: shuffle/skew (even vs uneven task times — the fork); spill = partition-count problem; broadcast vs sort-merge + stale stats; output file sizing; compaction; partition pruning + columnar → Athena $/TB; safe idempotent backfill.

Senior → Principal signal: Open the Spark UI mentally — is time spread evenly or is one task the straggler? That fork (skew vs sizing) drives the whole diagnosis (P06). Connect the through-line: the small files that slow the write are the same files inflating the Athena bill (P08→P10), and the fix is layout, not a bigger cluster.

Round 4 — Scala Platform Engineering

Prompt (JD): Design a Scala library so product teams define event schemas, validate data contracts, build stream processors, write to lakehouse tables, and get observability automatically.

Prepares it: P12 (Scala/Cats/Effects/SDK), P03 (contracts), P04 (sinks), P13 (observability).

What they probe: type-safe APIs (make illegal states unrepresentable); Validated (accumulate) vs Either (fail-fast); Resource safety; effect systems (referential transparency); error modeling; retry; binary compatibility + versioning; ergonomics.

Senior → Principal signal: The spine is "make misuse uncompilable" (P12). Newtypes + smart constructors for events; ValidatedNel decode → DLQ; effects (IO/ZIO) so processing is lazy/testable; Resource so connections can't leak; the lakehouse write as a single safe exactly-once API; semver + MiMa so upgrades never break consumers. "The happy path is the only path that compiles."

Round 5 — Architecture Leadership (Migration Review)

Prompt (JD): Lead a design review migrating legacy Hive/Tez/Flume to Kafka/Flink/Spark/ Iceberg/Athena.

Prepares it: P07 (legacy), P09 (Iceberg), P15 (migrations/ADRs/leadership), P13 (validation).

What they probe: migration phases; dual-write; data validation/parity; replay; cutover; rollback; cost; governance; team adoption; deprecation; risk management.

Senior → Principal signal: Strangler-fig, never big-bang (P15): dual-write → validate parity (data diffing, P13) → incremental cutover (old path warm) → decommission last, reversible at every step. Plus the leadership layer — present in cost/risk/time/reliability to leadership, set the standards, mention team adoption and the deprecation plan. The signal is patience + reversibility + business framing, not technical bravado.

Behavioral / Principal-Bar (woven through)

The JD's seniority bar: system-wide judgment, operating under ambiguity, deep debugging, influence without authority, written architecture communication, simplifying without making fragile, building platforms not pipelines, mentoring, setting standards, balancing the forces. Prepare stories that show: a decision you wrote as an ADR and revisited; a migration you made reversible; an incident you reduced to one root cause; a standard you set that scaled the team; a time you chose the boring, operable option over the clever one.

How to Practice

1. Take any phase's WARMUP.md Interview Q&A and answer aloud, with numbers.
2. Whiteboard each system-design walkthrough in 45 minutes.
3. For every tool choice, state the dial + deciding factor (P15) — out loud, then as an ADR.
4. Re-run the labs from memory; the flagship labs are past interview questions in disguise.