Phase 01 — Rack-Scale AI Systems & Data Center Foundations
Difficulty: ⭐⭐☆☆☆ | Estimated Time: 2 weeks Roles supported: Rack Management SWE (Senior/Staff), Data Center SWE, Infra/Platform Engineer Hardware needed: none — both labs run on any laptop
Why This Phase Exists
You cannot write software that manages a thing you cannot picture. Before a single Redfish call or Ansible play, you must be able to draw an AI rack from memory — every box, every cable, every power and cooling path — and reason about it as a system with budgets, failure domains, and a lifecycle. This is the mental model that the rest of the curriculum populates with code.
The JD's first line is "rack management software for AI accelerator products deployed in large-scale data center environments." That sentence hides an enormous amount of physical reality: an AI rack today can draw 40–130+ kW (a traditional enterprise rack drew 5–10 kW), is often liquid-cooled, contains its own management network and dozens of independently-firmwared components, and is the unit that gets provisioned, monitored, and maintained as a whole. Get the physical model wrong and every abstraction above it leaks.
This phase builds that base — not trivia, mechanism and numbers. By the end you can explain why an AI rack needs a CDU, why management traffic rides a separate network, what a "failure domain" means when a single PDU feeds eight nodes, and what fields your inventory database absolutely must have. You will encode that understanding as two pieces of working software: a rack data model and a power/thermal budget calculator.
Concepts
- What "rack-scale" means: the rack as the unit of deployment, sale, and management for AI systems; scale-up vs scale-out
- Rack anatomy: compute trays/nodes, AI accelerators, host CPUs, NICs/DPUs, scale-up fabric (NVLink/NVSwitch-class), top-of-rack (ToR) data switches, management switch, PDUs, busbar, PSU shelves, CDU
- The management plane: BMC (baseboard management controller) as "the computer inside the computer"; out-of-band (OOB) vs in-band; why they are physically separate networks
- Power: 3-phase power, PDUs and busbars, PSU shelves/redundancy (N+1, 2N), rack power density, power budgeting and capping, the difference between nameplate, rated, and measured power
- Cooling: air vs direct-liquid-cooling (DLC), CDU (coolant distribution unit), rear-door heat exchangers, thermal budgets, ΔT, PUE; why AI density forced the shift to liquid
- Units & physical facts: rack U / OpenRack OU, dimensions, weight, BTU/hr, kW↔BTU conversion
- Reference designs: OCP (Open Compute Project) & Open Rack, NVIDIA MGX / HGX / GB200 NVL72, DGX, hyperscaler in-house designs — and where Qualcomm AI accelerators fit
- Failure domains & blast radius: power domain, cooling domain, network domain, the rack as a correlated-failure unit; redundancy vs isolation
- The rack lifecycle: design → ODM build (Pegatron and peers) → integration/L10–L11 → ship → install → commission/burn-in → provision → operate → maintain/RMA → decommission
- The ecosystem: OEM vs ODM vs ISV; the software EBOM; DCIM (data center infrastructure management) and where rack management software sits relative to it
- What rack management software must model: inventory/asset schema, identity, relationships (contains/powers/cools/connects), state, and telemetry
Labs
Lab 01 — Rack Data Model & Inventory Engine (Python)
| Field | Value |
|---|---|
| Goal | Design and implement the core data model a rack manager is built on: a typed object graph of racks → trays → components, with relationships (powers, cools, connects), state, and a query/validation layer. |
| Concepts | Inventory/asset modeling, component identity (FRU, serial, asset tag), relationship graphs, state machines, validation invariants, why the model is the spine of everything. |
| Steps | 1) Read solution.py top-down. 2) Run it; study the rack it builds and the validation report. 3) Extensions: add a CDU + liquid-cooling loop to the model; add a find_blast_radius(component) query; serialize to/from JSON (the Redfish-shaped export). |
| Stack | Python 3 (stdlib + dataclasses), no external deps |
| Output | A working inventory engine that builds a realistic AI rack and reports invariant violations (e.g., a node with no power feed). |
| How to Test | python solution.py — built-in asserts verify the graph, the relationship queries, and the validation invariants. |
| Talking Points | Why inventory is the hardest "boring" problem in DCIM; identity vs location; how Redfish models the same graph (Chassis/Systems/Managers); what a "blast radius" query is for. |
| Resume Bullet | "Designed and implemented a typed rack inventory model (components, relationships, state, validation invariants) forming the data spine of a rack-management toolchain; added blast-radius and Redfish-shaped JSON export." |
| Extensions | Add temporal state (a component's history); add a reconciliation diff between "declared" and "discovered" inventory (the seed of Phase 06's reconcile loop). |
Lab 02 — Power & Thermal Budget Calculator (Python)
| Field | Value |
|---|---|
| Goal | Build the tool that answers "can this rack take one more node?" — sum component power against PDU/feed/breaker limits and heat load against CDU/cooling capacity, with redundancy (N+1, 2N) modeled. |
| Concepts | Power budgeting, breaker derating (the 80% rule), redundancy math, kW↔BTU↔cooling, ΔT and flow, power capping, headroom and alerting thresholds. |
| Steps | 1) python solution.py. 2) Read the budget report for a populated rack. 3) Push it over budget by adding nodes and watch the violations. 4) Extensions: model a PSU/feed failure and re-check redundancy; add per-node power capping to fit a constrained rack. |
| Stack | Python 3 (stdlib) |
| Output | A budget report: total draw vs limits, redundancy status, cooling headroom, and a pass/fail with reasons. |
| How to Test | Asserts verify the 80% breaker rule, N+1 redundancy loss detection, and the kW→BTU conversion. |
| Talking Points | Why nameplate ≠ measured power; the 80% continuous-load rule; why losing one PSU in an N+1 shelf is fine but two is not; how power capping trades performance for fit. |
| Resume Bullet | "Built a rack power/thermal budget calculator enforcing breaker derating and N+1/2N redundancy, with power-capping to fit density-constrained racks; used to validate rack configurations pre-deployment." |
| Extensions | Pull live draw from the Phase 04 PDU/CDU simulator and turn the calculator into a live headroom monitor; emit Prometheus metrics (foreshadows Phase 07). |
Deliverables Checklist
- You can draw an AI rack from memory: trays, accelerators, fabric, ToR, mgmt switch, PDU/busbar, PSU, CDU, BMC
- You can explain OOB vs in-band and why the management network is physically separate
- Inventory engine builds a realistic rack and reports invariant violations
- You can compute a rack's power budget with the 80% breaker rule and N+1 redundancy in your head for simple cases
- Budget calculator flags over-subscription and redundancy loss with reasons
- You can name the rack lifecycle stages and where rack-management software acts in each
- You can do the kW→BTU/hr conversion and explain why AI density forced liquid cooling
Interview Relevance
- "Walk me through the components of an AI rack and how they're powered and cooled."
- "What's the difference between the management network and the data network, and why does it matter for your software?"
- "A rack has two 3-phase 30A feeds. How much IT load can you safely put on it, and why not 100%?"
- "Design the data model for a rack inventory system. What are the entities and relationships?"
- "What's a failure domain? Give an example in a rack and how your software should treat it."
- "Why are AI racks liquid-cooled when enterprise racks weren't?"