🛸 Hitchhiker's Guide — Phase 15: Principal Architecture & Strategy

Read this if: you want the principal's decision/migration/leadership toolkit fast, and to pass Round 5 (lead the migration review). Skim, read WARMUP, build the toolkit.

0. The 30-second mental model

A principal's output is decisions, defended with numbers and made durable as ADRs. You choose tools by running a weighted matrix (not fashion), build only your differentiator, migrate via dual-write → validate → cutover → decommission (always reversible), and scale yourself through standards + reference architectures. One sentence: name the dial, quantify both ends, choose, write the ADR — and lead the migration without a big-bang.

1. The decision discipline

1. frame as a dial (P00 forces in conflict)
2. quantify both ends with the workload's numbers
3. score options on weighted factors → DECIDING FACTOR (the one that tipped it)
4. write the ADR: Context · Decision · Alternatives · Consequences · "what would reverse it"
ADR = decisions become legible, teachable, reversible-on-purpose

2. Build vs buy

TCO: Build = high upfront / low run ; Buy = low upfront / high run → find the CROSSOVER
non-cost: control, lock-in, time-to-value, can-you-operate-it, IS IT YOUR DIFFERENTIATOR?
heuristic: BUILD your differentiator; ADOPT everything commoditized (Kafka/Iceberg/…)
           build only the thin platform layer (SDK, P12) that encodes YOUR standards

3. Tool selection = run the matrix (you built the inputs)

Kafka/Kinesis/Pulsar (P02) · Flink/Spark-SS/Kafka-Streams/Akka (P04/05)
Iceberg/Delta/Hudi (P09) · Athena/Trino/Spark-SQL (P10) · Cassandra/DynamoDB (P11) · EMR×3 (P07)
"best tool" is meaningless without weights — the SAME two flip winners under different weights

4. Safe migration (Round 5)

dual_write → validate parity → cutover (incremental, old path warm) → decommission (LAST)
NEVER cut over before parity holds ; ALWAYS keep rollback ; never big-bang
= strangler-fig: grow new around old, shift load, retire old
same discipline as schema evolution (P03), backfills (P06), table adoption (P09)

5. Multi-region / DR

RPO (data you can lose) + RTO (time to recover) = the dial (more nines = more $)
active-passive (cheap, slower RTO) vs active-active (instant, costly, multi-write conflicts P01)
DR is only real if GAME-DAY tested (P14)

6. Scale yourself: standards + reference architectures

encode decisions as STANDARDS so teams don't re-decide (or re-mistake)
streaming standard (event-time+idempotency+DLQ), lakehouse standard (Iceberg+compaction), contracts
= platform-as-product: self-service WITH guardrails (P00)
→ "engineers create pipelines without reinventing architecture" (a JD success metric)

7. Design reviews & leadership

review = pressure-test the ADRs (dials? numbers? failure modes? rollback? sensitivity?)
to leadership: speak COST / RISK / TIME / RELIABILITY, not Flink internals
leadership: influence w/o authority, mentor, simplify-without-fragility, calm-from-numbers

8. Beginner mistakes that mark you

  1. "Best architecture" with no workload weights.
  2. Building a message bus / table format (not your differentiator).
  3. Big-bang migration with no parity validation or rollback.
  4. Decisions in chat, not ADRs → re-litigated every quarter.
  5. Presenting Flink internals to execs instead of cost/risk/time.
  6. A DR plan never game-day tested.
  7. Being the bottleneck (deciding everything) instead of setting standards.

9. War stories

  • "The migration broke prod." → big-bang cutover, no parity soak, no rollback.
  • "We re-argue Flink vs Spark every quarter." → no ADR captured the decision + revisit criteria.
  • "We built our own orchestrator." → not the differentiator; years of toil; adopt Dagster.
  • "Failover didn't work in the real outage." → never game-day tested.

10. How this phase pays off later

  • Decides between everything in P02–P14; leads the migration that composes them.
  • ADR discipline = P00's "name the dial," formalized; standards = the platform of P13.
  • Directly → the P16 capstone (compose it all) and your career trajectory.

Read WARMUP, build the toolkit, write some ADRs, then P16: compose the entire track into the five capstone systems — the JD's hard problems, end to end.