System Design — Apache Ecosystem Walkthroughs
Three full design walkthroughs at the altitude expected of a staff/principal candidate or a PMC-level design-doc author. Each follows the same discipline the Phase 12 design-doc capstone requires: problem → constraints → alternatives with rejection reasons → chosen design → failure modes → rollout/compatibility plan.
| Doc | Design problem | Trains |
|---|---|---|
| 01-yarn-scheduler-design.md | Multi-tenant cluster scheduler (YARN-style) | Resource arbitration, preemption, fairness vs utilization |
| 02-hive-metastore-scaling.md | Scaling a metadata service from thousands to millions of partitions | Hot metadata stores, caching, API-compatibility under load |
| 03-public-api-evolution.md | Evolving a public API used by thousands of downstream projects | Deprecation lifecycle, binary vs source compat, migration tooling |
How to use these
- Before reading each doc, spend 45 minutes designing it yourself on paper from the problem statement at the top. Then compare — the deltas are your study list.
- In interviews, the differentiator is not the final architecture but the order you surface concerns: workload characterization → consistency/compat constraints → the 2–3 real alternatives → failure modes — before drawing boxes.
- Each doc ends with interviewer follow-up questions. Practice answering them aloud.