Phase 15 — Principal Architecture & Strategy
Difficulty: ⭐⭐⭐⭐⭐ (judgment, not code) Estimated Time: 2 weeks (24–32 hours) Prerequisites: all prior phases (you decide between the things you now understand)
Why This Phase Exists
Everything before this gave you the tools; this phase is the job. The JD's "Lead Technical Strategy and Architecture Reviews" responsibility: "create architecture decision records, review major designs, define streaming and lakehouse standards, decide when to use Spark/Flink/ Hive/Athena/Trino/Cassandra/Kinesis/Kafka/EMR, evaluate new technologies, lead migrations away from obsolete systems, mentor senior engineers, raise the bar, create reference architectures, present tradeoffs to leadership." Round 5 is exactly this: lead the Hive/Tez/Flume → Kafka/ Flink/Spark/Iceberg/Athena migration review. This phase teaches how a principal decides, migrates, and leads — and how to make those decisions durable and defensible.
Concepts
- The decision discipline (P00, formalized): name the dial, quantify both ends, choose, write the ADR (context, decision, alternatives, consequences, what would reverse it).
- Build vs buy: TCO over a horizon (Build = high upfront/low run; Buy = reverse; the crossover), plus the non-cost factors (control, lock-in, time-to-value, team skill, differentiation — don't build what isn't your differentiator).
- Tool selection as ADRs: when Spark vs Flink vs Spark-SS; Kafka vs Kinesis vs Pulsar; Iceberg vs Delta vs Hudi; Athena vs Trino; Cassandra vs DynamoDB — each a weighted decision with a deciding factor, not a fashion.
- Migrations (Round 5): the safe pattern — dual-write → validate parity → cutover → rollback-ready → decommission (the lab's gate checker); strangler-fig incrementalism; data validation/diffing (P13); never a big-bang; always reversible.
- Multi-region architecture & DR: RPO/RTO, active-active vs active-passive, replication (P14), consistency tradeoffs (P01).
- Reference architectures & standards: defining the org's blessed patterns (the streaming template, the lakehouse table standard, the contract policy) so teams don't reinvent — the platform-as-product mindset (P00).
- Design reviews: running them (pressure-test the ADRs), and presenting tradeoffs to leadership in their language (cost, risk, time, reliability).
- Technical leadership: mentoring senior engineers, raising the bar, influence without authority, simplifying without making fragile, operating as a principal.
Labs
Lab 01 — Architecture Decision Toolkit (flagship, implemented)
| Field | Value |
|---|---|
| Goal | Build a weighted decision matrix with deciding factor, TCO, a safe-migration gate checker, RPO/RTO check, and an ADR renderer |
| Concepts | Decision matrices, TCO, safe-migration ordering, RPO/RTO, ADRs |
| How to Test | pytest test_lab.py -v — 15 tests |
| Talking Points | Why is "best architecture" meaningless without weights? What makes a migration safe? When build vs buy? |
| Resume bullet | Built an architecture-decision toolkit (weighted matrices, TCO, safe-migration gating, RPO/RTO, ADR generation) to drive and document platform strategy |
→ Lab folder: lab-01-decision-toolkit/
Extension Project A — Write 3 real ADRs (spec)
Pick three decisions from this track (table format, streaming engine, serving store) and write full ADRs with weighted matrices and the deciding factor — your portfolio of judgment.
Extension Project B — Lead the Round-5 migration review (spec)
Produce the full Hive/Tez/Flume → Kafka/Flink/Iceberg/Athena migration plan: phases, dual-write, validation strategy (data diffing, P13), cutover, rollback, cost, governance, team adoption, deprecation — the design-review deck.
Integrated-Scenario Hooks
- This phase decides between everything in P02–P14, and leads the migration that composes them — the bridge to the P16 capstone.
- Its ADR discipline is the through-line from P00's "name the dial."
Guides in This Phase
Key Takeaways
- A principal's output is decisions, defended with numbers and made durable as ADRs.
- "Best architecture" is meaningless without the workload's weights; name the deciding factor.
- Build only your differentiator; buy/adopt the rest (TCO + lock-in + time-to-value).
- Migrations are dual-write → validate → cutover → decommission, always reversible, never big-bang.
- Leadership = standards, reference architectures, design reviews, mentoring, influence.
Deliverables Checklist
- Lab 01 implemented; all 15 tests pass
- You can produce a scored ADR for a real decision
- You can lay out a safe migration with correct gates and rollback
- You can present a tradeoff to leadership in cost/risk/time terms
- Extension A three ADRs or Extension B migration review deck