🛸 Hitchhiker's Guide — Phase 13: The Capstone

Read this if: you've done the phases and need to assemble them into one platform you can demo and defend. Compressed map of the composition.

0. The 30-second mental model

Compose the parts into a control plane over a driver abstraction, held together by three ideas: one data model (P01), one reconcile loop (P06), clean seams (the DeviceDriver, P03 — the testability superpower). Provisioning (P05) and firmware (P08) are actions the loop takes, not separate orchestrators. Ship it with a design doc + runbook + retro (P12). Demo the failure paths, not just the happy path.

1. The architecture

rackctl (P02) → control plane:
   inventory model (P01)  ←—  reconcile loop (P06): declared → discovered, converge
   provisioning SM (P05) + EBOM gate (P12) + attestation (P09)
   firmware A/B + rollback (P08)
   telemetry/exporter + alerts (P07)
   tenancy/secure config + isolation invariants (P09)
        ↓ DeviceDriver abstraction (P03): Redfish-first, IPMI/SNMP fallback, FakeDriver for tests

2. The three load-bearing ideas

  • one model (P01): the spine every part reads/writes; declared baseline = EBOM.
  • one reconcile loop (P06): idempotent, resumable, level-triggered convergence.
  • clean seams: the driver boundary (swap real BMC ↔ fake), the model, the reconcile API.

3. Lifecycle as states

DISCOVERED → INVENTORIED(+EBOM) → FIRMWARE_BASELINED → ATTESTED → OS_INSTALLED → CONFIGURED → VALIDATED → READY → operate(telemetry/drift-correct) → maintain(firmware/RMA) → DECOMMISSION. Idempotent · resumable · validation-gated · observable · reconciled vs intent.

4. Demo the failures (what proves it's real)

flaky BMC → driver retry+breaker, loop requeues · bad node → quarantine w/ reason (EBOM/attest/RAS) · bad firmware → A/B rollback + canary halt · correlated failure → blast-radius RCA (P01) · control plane restart → idempotent/persisted, no corruption.

5. What changes at 10k nodes (the design-doc section)

telemetry → federate/remote-write + cardinality (P07, sd/02) · control plane → shard + smart proxies (sd/01) · provisioning/firmware → progressive, domain-aware, canary (sd/03) · tenancy → isolation + attestation at scale (sd/04) · ops → SLOs + cross-site on-call (P07/12).

6. Ship like an engineer

design doc (architecture + seams + ADRs + failure handling + fleet section) · runbook (P12) · release via the pipeline (P11) + a retrospective (your own RCA-discipline on your own work, P10).

7. The 5-minute tour to rehearse

"control plane over a driver abstraction; one model, one reconcile loop, clean seams; lifecycle as states; here's a flaky BMC handled, a node quarantined, a firmware rolled back; at 10k nodes I'd shard + federate + canary; design doc + runbook + retro attached." Nail that and the loop is yours.

8. How it ties the whole track together

The capstone is the portfolio behind every interview answer. It reuses P01 (model), P02 (CLI/ patterns), P03 (drivers), P05 (provisioning), P06 (reconcile), P07 (telemetry), P08 (firmware), P09 (tenancy), P10 (reliability), P11 (tests/release), P12 (docs) — and points at system-design/ for scale.