Warmup Guide — Rack-Scale AI Systems & Data Center Foundations

Zero-to-expert primer for Phase 01. You need no data-center background: this guide starts at "what is a rack unit" and ends with you sizing power and cooling for a 120 kW AI rack and explaining why your management software needs a separate network. Every later phase — Redfish, BMC, provisioning, telemetry, firmware — stands on this physical model.

Table of Contents

Chapter 1: Why "Rack-Scale" Is the Unit That Matters

Zero background: a rack is a standardized steel cabinet (19-inch wide mounting rails, historically ~42U tall, where 1U = 1.75 inches) that holds servers and networking gear. For thirty years the server was the unit of everything — you bought a server, provisioned a server, monitored a server. The rack was just furniture.

The shift: AI training and inference need many accelerators wired together with very fast links (a "scale-up" fabric like NVLink/NVSwitch). Those links only reach a few meters at full bandwidth, so the natural boundary is one rack. Inside a rack, accelerators talk at ~hundreds of GB/s to TB/s; between racks, they drop to network speeds (tens of GB/s). That bandwidth cliff makes the rack the natural unit of design, sale, deployment, and management. NVIDIA sells a GB200 NVL72 as a rack. Hyperscalers deploy and retire capacity in racks.

Two words you must separate:

  • Scale-up = make one coherent machine bigger by tightly coupling accelerators (NVLink fabric inside a rack). Latency/bandwidth-sensitive; bounded by physics to ~a rack.
  • Scale-out = add more independent machines connected by a network (Ethernet/InfiniBand across racks). This is the data-center-wide dimension.

Why this matters for your software: rack-management software treats the rack as a managed object with sub-components, a power/cooling budget, a single management network, and a correlated failure profile. Your inventory model (Lab 01), your provisioning unit (Phase 05), and your blast-radius reasoning (Chapter 7) all key off "the rack." When the JD says "lifecycle management of rack-scale AI systems," rack-scale is the literal scope of the object you manage.

Common misconception: "a rack is just N servers, manage each server." For AI, the rack has shared resources (power shelf, busbar, CDU, NVSwitch trays, ToR) that belong to no single server and must be managed as rack-level objects. Miss them and you can't model power, cooling, or the scale-up fabric at all.

Chapter 2: Anatomy of an AI Rack, Box by Box

Picture a modern liquid-cooled AI rack from top to bottom. Memorize this list — Lab 01 turns it into a data model.

  • Compute trays / nodes (the bulk of the rack): each tray holds host CPUs, system memory, AI accelerators (GPUs / Qualcomm Cloud AI / TPUs / custom ASICs), local NVMe, and NICs. A tray is one or more "nodes" (a node = one OS instance = one BMC, usually).
  • AI accelerators: the reason the rack exists. 4–8 per tray is common. Each has its own high-bandwidth memory (HBM), firmware, and thermal profile, and is the hottest, most expensive, most-monitored component.
  • Host CPU(s): run the OS, the in-band agents, and feed the accelerators. 1–2 sockets per node.
  • NICs / DPUs: the data-plane network interface. AI nodes often have multiple high-speed NICs (one per accelerator for RDMA) plus a separate management NIC. A DPU (SmartNIC) is a NIC with its own CPU/OS that can offload networking, storage, and security — itself a managed device with firmware.
  • Scale-up fabric: NVSwitch trays (or vendor equivalent) that wire accelerators into one coherent fabric. In an NVL72 these are dedicated trays with their own management.
  • Top-of-Rack (ToR) data switch: the scale-out Ethernet/InfiniBand switch connecting this rack's nodes to the rest of the data center.
  • Management switch: a separate, smaller switch carrying only out-of-band management traffic (BMCs, PDUs, CDU controller) — see Chapter 3/6.
  • Power shelf / PSUs: AC→DC conversion. In OCP racks, a power shelf feeds a vertical DC busbar that runs the height of the rack; nodes tap the busbar with blind-mate connectors. In legacy racks, each server has its own PSUs fed by PDUs (power distribution units — essentially smart power strips).
  • PDU(s): distribute and (if "smart"/metered/switched) measure and control power per outlet or per phase. A managed device you talk to over SNMP/Redfish (Phase 03/04).
  • CDU (Coolant Distribution Unit): for liquid cooling — pumps coolant through the rack's cold plates, exchanges heat with facility water, and reports flow/temperature/pressure. A managed device (Chapter 5).
  • BMC per node (and often a rack manager / RMC): the always-on management microcontrollers (Chapter 3).

A useful sanity image: an NVL72 rack ≈ 18 compute trays (2 Blackwell GPUs + 1 Grace CPU each → 72 GPUs) + 9 NVSwitch trays + power shelves + busbar + manifold/CDU plumbing + ToR + management switch. ~1.36 metric tons, ~120 kW. That is "rack-scale."

Chapter 3: The Management Plane — BMC, OOB vs In-Band

This is the single most important concept for your job, so go slow.

The BMC (Baseboard Management Controller) is a small, independent computer soldered onto the server's motherboard — typically an ARM SoC (e.g., ASPeed AST2600) running its own Linux (often OpenBMC) with its own memory, its own network port, and its own power rail (standby power). It is the computer inside the computer. It runs even when the host is powered off (as long as the rack has standby power).

What the BMC does:

  • Power control: power on/off/cycle the host, regardless of the host OS state.
  • Sensors: read temperatures, voltages, fan speeds, power draw — independent of the host.
  • Console: Serial-over-LAN (SoL) and KVM (remote keyboard/video/mouse) so you can watch the host boot.
  • Virtual media: mount an ISO over the network so you can reinstall the OS remotely.
  • Logs: the System Event Log (SEL) and crash/RAS captures (Phase 08).
  • Firmware: it can update host firmware (BIOS/UEFI) and its own.

Out-of-band (OOB) vs in-band — the distinction your whole control plane rests on:

  • In-band = management that goes through the host OS and its main NIC: SSH to the host, an agent reading /proc, the data NIC. Works only when the host is up and healthy.
  • Out-of-band = management that goes through the BMC over the management network, independent of the host OS. Works when the host is off, hung, kernel-panicked, or mid-reinstall.

Why OOB exists / why you cannot live without it: the moments you most need to manage a machine are exactly the moments it's not healthy — it won't boot, it kernel-panicked, it needs an OS reinstall. In-band management is useless then. OOB (Redfish/IPMI to the BMC, Chapter 6 / Phase 03) is how you power-cycle a wedged node, watch why it won't POST, and reinstall it — from a continent away. The JD's "remote diagnostics," "firmware updates," and "support of deployed systems" all live on the OOB plane.

Mental model: every node is really two computers on two networks — the host (in-band, data network) and the BMC (out-of-band, management network). Rack management software spends most of its time talking to the second one.

Misconception: "the BMC is part of the OS." No — it's separate silicon with separate firmware and power. You can redfish GET a node's sensors while its host OS is completely dead. That independence is the entire point.

Chapter 4: Power — From the Feed to the Chip

Power is where rack-scale AI breaks everything that worked at enterprise scale. You must be able to do this math (Lab 02).

The path: utility → data hall → 3-phase AC to the rack → PDU/power-shelf → PSUs (AC→DC) → busbar/cables → node → VRMs → chip.

3-phase power, minimally: data centers distribute 3-phase AC because it carries more power per conductor. A common rack feed is, e.g., 3-phase 415V 30A. Usable power ≈ √3 × V_line-to-line × I × power-factor ≈ 1.732 × 415 × 30 × ~1 ≈ 21.6 kW per feed (regional voltages vary; the point is the formula). Two feeds for redundancy → ~43 kW available, but see the 80% rule.

The 80% rule (NEC continuous-load derating): a breaker rated for continuous load may only be loaded to 80% of its rating. A 30A circuit → budget to 24A. This is not optional; exceed it and the breaker trips (an outage) or you create a fire risk. Every power-budget calculation a rack manager does derates by 0.8. Lab 02 enforces this.

Redundancy schemes (memorize):

  • N: just enough capacity, no spare. One failure = outage.
  • N+1: one spare unit; survives a single PSU/feed failure. The common default.
  • 2N: fully duplicated (two independent feeds, each able to carry the whole load). Survives a whole side failing. Used for the most critical gear.

The trap: in N+1, the load must fit on N units after one fails. If you fill the rack assuming N+1 capacity, a single failure overloads the survivors and cascades. Lab 02's redundancy check is exactly this.

Power density, then and now:

  • Enterprise rack (2010s): 5–10 kW.
  • Dense compute: 15–20 kW.
  • AI air-cooled: 30–40 kW (near the limit of what air can remove).
  • AI liquid-cooled (NVL72-class): 60–130+ kW per rack, heading toward 250 kW–1 MW in next-gen designs.

That ~10× jump is the reason this role exists: at 120 kW, power and cooling are first-class software-managed budgets, not facility afterthoughts.

Nameplate vs rated vs measured:

  • Nameplate (TDP/max): the worst-case the component could draw. Sizing breakers naively to nameplate strands huge capacity.
  • Rated/derated: what you provision for after derating.
  • Measured (telemetry): actual draw, read from PDU/BMC (Phase 04/07). Almost always well below nameplate.

Power capping: the BMC/accelerator can be told to cap power (clock/voltage throttle) to fit a budget — trading performance for thermal/power fit. Rack management software uses capping to (a) safely oversubscribe based on measured behavior and (b) ride out a cooling/power degradation without a hard outage. Lab 02's extension implements capping-to-fit.

Chapter 5: Cooling — Why AI Racks Went Liquid

Every watt of power becomes a watt of heat. A 120 kW rack produces 120 kW of heat that must leave the building, or the chips throttle and then die.

The conversion to know cold: 1 kW = 3,412 BTU/hr. A 120 kW rack dumps ~409,000 BTU/hr. For intuition: that's roughly the heat of ~34 home ovens running flat-out, in one cabinet.

Why air stops working: air has low heat capacity; removing 30–40 kW from a rack already requires gale-force airflow and chilled aisles. Beyond ~40 kW, air physically can't keep modern accelerators below their throttle temperature. So the industry moved to liquid.

Direct Liquid Cooling (DLC): coolant runs through cold plates bolted directly onto the hot chips (accelerators, sometimes CPUs/VRMs). Liquid carries ~3,500× the heat per volume of air. Variants: cold-plate (most common), rear-door heat exchangers (a radiator on the rack's back door), and full immersion (the whole board in dielectric fluid — rarer).

The CDU (Coolant Distribution Unit): the heart of a liquid-cooled rack. It:

  • pumps the secondary (rack-side) coolant loop through the cold plates,
  • exchanges heat with the facility's primary (chilled-water) loop via a heat exchanger (keeping the clean rack loop isolated from facility water),
  • reports flow rate (L/min), supply/return temperatures, ΔT, pressure, pump speed, and leak detection.

That last bullet is yours: the CDU is a managed device (Redfish/Modbus/SNMP) your software monitors. A failing pump or a rising ΔT is an early warning of a thermal event; leak detection is a safety-critical alert that may trigger an emergency drain/shutdown.

Thermal budget math (Lab 02): heat removed = flow × fluid heat capacity × ΔT (supply vs return temperature). If component heat load exceeds the CDU's rated removal at safe ΔT, the rack overheats. Your calculator checks heat load against cooling capacity exactly as it checks power against breakers.

PUE (Power Usage Effectiveness): total facility power ÷ IT power. 1.0 is perfect (all power does compute). Air-cooled enterprise: ~1.5–2.0. Liquid-cooled AI halls push toward ~1.1. PUE is a facility KPI, but rack-level cooling efficiency feeds it — and "tokens per joule" / "performance per watt" is increasingly how AI infra is judged.

Misconception: "cooling is the facility team's problem." At 120 kW with liquid loops, cooling telemetry (CDU flow/ΔT/leak) is part of the rack's health that your software monitors and acts on. A leak or pump failure is a software-visible, software-actionable event.

Chapter 6: Networks — Three of Them, Not One

A common beginner mistake is imagining "the network." An AI rack has (at least) three logically separate networks, and conflating them is a security and reliability bug.

  1. Data / compute network (in-band, scale-out): high-speed Ethernet or InfiniBand (e.g., 400/800 GbE, NDR IB) connecting nodes to the ToR and out to the cluster. Carries the actual AI traffic (gradients, activations, RDMA). Highest bandwidth, tenant-visible.
  2. Scale-up fabric (intra-rack): NVLink/NVSwitch-class links wiring accelerators into one coherent memory fabric. Not "Ethernet"; managed as part of the rack's hardware.
  3. Management network (out-of-band): a separate, modest (often 1 GbE) network connecting every BMC, PDU, CDU, and managed switch to the management/control plane. This is where Redfish/IPMI/SNMP live. It is physically and administratively isolated from tenant data.

Why management is separate (this is an interview favorite):

  • Reliability: you must reach the BMC to fix a node even when its data network/host is down. If management rode the data network, a data-network outage would also blind your control plane — exactly when you need it.
  • Security: BMCs are extraordinarily powerful (full power and firmware control) and historically under-hardened. Exposing them on the tenant network would be catastrophic. They live on an isolated management VLAN reachable only by the control plane (Phase 09).
  • Blast-radius: separating planes means a problem in one doesn't take out the other.

The JD's "VLAN separation" (Phase 09) is, in large part, about keeping these planes — and different tenants within the data plane — isolated. Hold this thought; Phase 09 builds it for real.

Chapter 7: Failure Domains & Blast Radius

A failure domain is a set of components that fail together because they share something. The blast radius of a fault is everything its failure domain contains. Designing and operating racks is largely the art of knowing and bounding blast radius — and your inventory model (Lab 01) must encode it.

Failure domains in a rack:

  • Power domain: everything fed by one PDU/feed/breaker. If a 24 kW feed dies and it powered 6 nodes without redundancy, those 6 nodes drop together. (N+1/2N exists to shrink this.)
  • Cooling domain: everything cooled by one CDU/loop/manifold. A pump failure or leak can take the whole rack's liquid-cooled components thermal at once.
  • Network domain: everything behind one ToR or one management switch. A ToR failure isolates the whole rack's data plane; a management-switch failure blinds your OOB control of the whole rack.
  • The rack itself: shared roof, shared facility water valve, shared row PDU — correlated failures up the hierarchy (rack → row → hall → site).

Why this matters operationally: workload placement and maintenance must respect domains. You don't put all replicas of a service on nodes sharing one PDU. You drain a whole power domain before maintenance on its feed. When something fails, the first diagnostic question is "what's the failure domain?" — a single node, or a shared resource taking many nodes with it? Lab 01's find_blast_radius(component) query is the software form of this question, and it's the seed of correlation logic your telemetry (Phase 07) and RCA (Phase 10) will need.

Redundancy vs isolation (don't conflate): redundancy (N+1) lets a domain survive a failure; isolation (separate domains, VLANs, MIG-style partitioning) contains a failure or a tenant. You design for both.

Chapter 8: Reference Designs — OCP, MGX, NVL72, and Where Qualcomm Fits

You don't need to memorize every SKU, but a leader/engineer in this space speaks the reference-design vocabulary.

  • OCP (Open Compute Project): the open-hardware community (started by Meta) that standardized hyperscale rack design: Open Rack (a wider 21-inch equipment space, 48 "OU"), the busbar/power-shelf model, and open management specs. Much of "modern rack" thinking is OCP-derived. Redfish, the OpenBMC firmware, and OCP's hardware management specs are the standards your software targets.
  • NVIDIA HGX / DGX / MGX: HGX is the baseboard of 8 GPUs + NVSwitch; DGX is NVIDIA's complete server; MGX is a modular reference architecture OEMs/ODMs build to. GB200 NVL72 is the rack-scale design (72 Blackwell GPUs as one NVLink domain, liquid-cooled, ~120 kW) — the canonical "AI rack" today.
  • Hyperscaler in-house: Google TPU pods, AWS Trainium/Inferentia racks, Meta's OCP-based designs, Microsoft Maia — each a rack-scale system with its own management software (the kind you'd build).
  • Where Qualcomm fits (this JD): Qualcomm's data-center AI accelerators (the Cloud AI 100 line and its successors) deploy in rack-scale systems. This role is the software that provisions, monitors, and maintains those racks — interfacing with the BMCs, PDUs, CDUs, and PCIe switches of accelerator products, integrating with orchestration and telemetry, and co-developed with ODM partners (the JD names Pegatron). The accelerator is the payload; your software runs the rack around it.

The strategic insight: hardware vendors increasingly win or lose on the software that operates the hardware at scale. A great accelerator with a flaky rack-management stack is unsellable to a hyperscaler. That's why this role is senior/staff and why the JD stresses production debugging and operational excellence.

Chapter 9: The Rack Lifecycle and the Ecosystem (OEM/ODM/ISV)

Rack management software acts across a rack's entire life. Know the stages and the players.

The ecosystem:

  • OEM (Original Equipment Manufacturer): sells the branded system (e.g., NVIDIA, Dell, HPE — or, for accelerators, Qualcomm).
  • ODM (Original Design Manufacturer): designs/builds the hardware, often white-label, for OEMs and hyperscalers. Pegatron (named in the JD), Quanta, Wiwynn, Foxconn are ODMs. You co-define interfaces, the software EBOM, and protocol behavior with them (Phase 12).
  • ISV (Independent Software Vendor): third-party software (orchestrators, DCIM, monitoring) you integrate with.

The lifecycle (and where your software acts):

  1. Design / NPI (New Product Introduction): define hardware + the management interfaces (Redfish schemas, sensor lists, the EBOM). Staff engineers contribute here (Phase 12).
  2. Manufacture & integration (L6–L11): the ODM builds boards (L6), nodes (L10), and integrates the full rack (L11), running factory tests. Your software may provide the test/validation tooling.
  3. Ship & install: rack arrives, is bolted down, powered, plumbed (liquid), and cabled.
  4. Commission / burn-in: discovery (find every BMC/PDU/CDU on the management network), inventory capture, firmware baseline, stress/burn-in tests, validation against the EBOM. This is rack-management software's first big job.
  5. Provision: install OS/firmware, configure networking, join the orchestrator (Phase 05/06).
  6. Operate: monitor (Phase 07), enforce tenancy/QoS (Phase 09), serve workloads.
  7. Maintain: firmware updates (Phase 08), diagnostics/RCA (Phase 08/10), component RMA (replace a failed accelerator/PSU/pump and re-validate).
  8. Decommission: secure wipe, asset retirement, update inventory.

Why a Staff engineer cares about the whole arc: bugs and design gaps surface at the seams between stages (e.g., a firmware baseline mismatch found only at burn-in). Owning "implementation through deployment and debugging in real-world conditions" (the JD's words) means your code and your mental model span this entire lifecycle.

Chapter 10: What Rack-Management Software Must Model

Synthesize the above into the thing you build first (Lab 01): the model. Get this right and everything composes; get it wrong and every feature fights the schema.

Entities (the nouns):

  • Site → Hall → Row → Rack → Tray/Node → Component (accelerator, CPU, NIC/DPU, NVMe, PSU, fan, BMC) — a containment hierarchy.
  • Shared rack-level objects: PDU, Busbar, PowerShelf, CDU, ToRSwitch, MgmtSwitch, NVSwitchTray.

Identity (how you name a thing, durably):

  • Asset tag (your inventory ID), serial number (manufacturer), FRU (Field Replaceable Unit) data, MAC/UUID, and location (rack + U position). Critical distinction: identity ≠ location — a node keeps its serial when it moves U positions or racks; your model must track both and reconcile them (a swapped-but-not-updated location is a classic data-integrity bug).

Relationships (the edges — this is what makes it a graph, not a table):

  • contains (rack contains node), powers (PDU powers node), cools (CDU cools node), connects (ToR connects node), manages (BMC manages node). These edges are what find_blast_radius traverses.

State:

  • Power state (on/off/standby), health (ok/warning/critical), lifecycle state (discovered/provisioning/ready/maintenance/decommissioned), firmware versions per component.

Telemetry (time-series, attached to components): temperature, power, fan/pump speed, errors — fed by Phases 04/07.

Invariants (validation rules your model enforces — Lab 01):

  • Every node has at least one power feed and (if liquid) a cooling path.
  • Power draw ≤ derated budget (links to Lab 02).
  • No two components claim the same U position.
  • Every managed device has a reachable management address.

Declared vs discovered: your model holds the declared intent (the EBOM / what should be there) and the discovered reality (what the management network actually found). The diff between them is gold: missing components, firmware drift, mislabeled locations. This declared-vs-discovered reconciliation is the conceptual seed of the Kubernetes operator pattern in Phase 06 and the RCA work in Phase 10.

Lab Walkthrough Guidance

Order: Lab 01 → Lab 02. Lab 01 builds the model (the nouns and edges); Lab 02 reasons over it (the budgets). Don't invert — the calculator consumes the model.

Lab 01 (Rack inventory engine):

  1. Read solution.py top-down before running; the dataclasses are Chapter 10. Predict what the validation report will flag.
  2. Run it; check the rack it builds against your mental Chapter-2 picture.
  3. Extensions in order: (a) add a CDU + cools edges (Chapter 5); (b) implement find_blast_radius(component) by traversing powers/cools/connects edges (Chapter 7); (c) JSON export shaped like Redfish's Chassis/Systems tree (foreshadows Phase 03).

Lab 02 (Power/thermal budget):

  1. python solution.py — read the budget report for the populated rack.
  2. Add nodes until you trip the 80% breaker rule; predict the failing feed before the tool tells you.
  3. Extensions: (a) fail one PSU in an N+1 shelf and confirm the survivors stay within budget; fail two and watch it go critical; (b) apply per-node power capping to fit a rack that's 10% over budget — the performance/fit tradeoff (Chapter 4).

The capstone question for the phase: "A partner proposes 18 trays at 6.5 kW each in a rack with two 30A 415V feeds, N+1, liquid-cooled with a 130 kW CDU. Does it fit — power and thermal — and what's the blast radius of one feed failing?" If you can answer that with numbers, Phase 01 did its job.

Success Criteria

You're done with this phase when — without notes:

  • You can draw an AI rack and name every box and which of the three networks each sits on (Ch. 2, 6)
  • You can explain OOB vs in-band and give two reasons the management network is separate (Ch. 3, 6)
  • You can size a 3-phase feed, apply the 80% rule, and check N+1 redundancy in your head (Ch. 4)
  • You can convert kW→BTU/hr and explain why >40 kW forced liquid cooling and what a CDU does (Ch. 5)
  • You can define a failure domain and compute a simple blast radius (Ch. 7)
  • You can name the rack lifecycle stages and what rack-management software does in each (Ch. 9)
  • Your inventory engine builds a valid rack, flags invariant violations, and answers a blast-radius query (Lab 01)
  • Your budget calculator enforces derating + redundancy and supports capping-to-fit (Lab 02)

Interview Q&A

Q1: Walk me through the components of an AI rack and how it's powered and cooled. A: Top-level, the rack holds compute trays (each with host CPUs, 4–8 AI accelerators with HBM, local NVMe, and multiple NICs/DPUs), scale-up fabric trays (NVSwitch-class) wiring the accelerators into one coherent fabric, a top-of-rack data switch for scale-out, and a separate management switch. Power comes in as 3-phase AC to a power shelf or PDUs; in OCP designs a power shelf feeds a vertical DC busbar that nodes blind-mate onto, with N+1 or 2N PSU redundancy. Because these racks draw 60–130+ kW — about 10× a legacy rack — they're liquid-cooled: a CDU pumps coolant through cold plates on the chips and exchanges heat with facility water, reporting flow, ΔT, pressure, and leak status. Every node also has a BMC on the management network. The key framing: it's not "N servers," it's a system with rack-level shared resources (power, cooling, fabric) that I manage as first-class objects.

Q2: What's the difference between out-of-band and in-band management, and why does it matter for your software? A: In-band management goes through the host OS and its data NIC — SSH, an agent reading /proc — and only works when the host is healthy. Out-of-band goes through the BMC, a separate always-on microcontroller with its own NIC on a separate management network, and works even when the host is off, hung, or kernel-panicked. It matters because the moments you most need to manage a node — won't boot, panicked, needs reinstall — are exactly when in-band is dead. So rack-management software does power control, firmware updates, console access, and diagnostics over OOB (Redfish/IPMI to the BMC). My control plane assumes the host may be down and reaches everything through the management network.

Q3: A rack has two 3-phase 415V 30A feeds, configured N+1. How much IT load can you safely place, and why not 100%? A: Per feed, usable ≈ √3 × 415 × 30 ≈ 21.6 kW, but the 80% continuous-load rule (NEC) caps a breaker at 80% of rating, so budget ≈ 17.3 kW per feed. With N+1, the surviving feed must carry the whole load after one fails, so the safe IT load is bounded by a single derated feed ≈ 17.3 kW — not the sum. I never budget to the combined or the nameplate capacity: exceed 80% and the breaker trips (an outage); budget to the sum and a single feed failure overloads the survivor and cascades. Measured draw is usually well below nameplate, so I can oversubscribe carefully using telemetry and power capping, but the guaranteed safe number is the derated single-feed figure.

Q4: Why are AI racks liquid-cooled when enterprise racks weren't, and what does that add to your job? A: Power density. Enterprise racks ran 5–10 kW, removable by air. AI racks run 60–130+ kW, and air physically can't remove that much heat past ~40 kW — 1 kW is 3,412 BTU/hr, so a 120 kW rack dumps ~409,000 BTU/hr. Liquid carries thousands of times more heat per volume, so coolant runs through cold plates on the chips, with a CDU pumping it and exchanging heat with facility water. For my software it adds a whole managed subsystem: I monitor CDU flow, supply/return ΔT, pressure, pump health, and leak detection; a rising ΔT or failing pump is an early thermal-event warning, and a leak is a safety-critical alert that may trigger emergency shutdown. Cooling becomes a software-visible, software-actionable part of rack health, not just a facility concern.

Q5: What's a failure domain? Give an example and how your software should treat it. A: A failure domain is a set of components that fail together because they share a resource; the blast radius is everything in it. Example: six nodes fed by one non-redundant PDU/breaker — if that feed dies, all six drop simultaneously. My software treats domains in three ways: (1) the inventory model encodes the powers/cools/connects edges so I can compute blast radius; (2) placement respects domains — don't put all replicas of a service behind one PDU or one ToR; (3) during incidents, the first question is "single node or shared resource?" — correlating alerts by domain turns a flood of node alarms into one root cause. Redundancy (N+1) lets a domain survive a failure; isolation (separate domains/VLANs) contains it — I design for both.

Q6: Design the data model for a rack inventory system. What are the entities, relationships, and the hard parts? A: Entities are a containment hierarchy — Site→Hall→Row→Rack→Node→Component — plus rack-level shared objects (PDU, busbar/power shelf, CDU, ToR, management switch, NVSwitch trays). Relationships are a graph, not just containment: powers, cools, connects, manages — those edges drive blast-radius queries. Each thing needs durable identity (asset tag, serial, FRU, MAC/UUID) kept separate from location (rack + U), because identity persists when a unit moves and a stale location is a classic bug. State covers power, health, lifecycle stage, and per-component firmware versions; telemetry is time-series attached to components. The hard parts: (1) identity-vs-location reconciliation; (2) declared-vs-discovered diffing — the model holds intent (the EBOM) and reality (what discovery found), and the diff catches missing parts and firmware drift; (3) validation invariants — every node has power and cooling, no two parts share a U, every managed device is reachable. That declared-vs-discovered reconcile is conceptually the same loop a Kubernetes operator runs.

Q7 (Staff): How would you approach commissioning a brand-new rack from an ODM partner? A: Commissioning is discovery → inventory → baseline → validate → hand-off. First, on the isolated management network, discover every managed endpoint (BMCs, PDUs, CDU, switches) — typically via DHCP + a Redfish/IPMI sweep. Capture full inventory (serials, FRU, firmware versions, accelerator topology) and reconcile it against the declared EBOM agreed with the ODM (e.g., Pegatron) — any missing/extra/mismatched part is a finding raised back to them. Set the firmware baseline (Phase 08) so the whole rack is at known-good versions. Run burn-in/stress and validate power and thermal behavior against budget (Phase 01 calculator) and the spec. Capture a golden inventory + telemetry baseline so future drift is detectable. Only then mark the rack "ready" and hand it to provisioning. The Staff value-add is making this repeatable and automated — a commissioning pipeline with a clear pass/fail and a documented escalation path to the partner — because at fleet scale you commission racks continuously, not once.

References

  • Open Compute Project — Open Rack & hardware management specs — https://www.opencompute.org/
  • DMTF Redfish — Schema & Specification (the management data model you'll meet in Phase 03) — https://www.dmtf.org/standards/redfish
  • NVIDIA GB200 NVL72 — architecture overview — https://www.nvidia.com/en-us/data-center/gb200-nvl72/
  • ASHRAE TC 9.9 — Thermal Guidelines for Data Processing Environments (cooling/ΔT references)
  • Uptime Institute — Tier topology & redundancy (N, N+1, 2N) concepts — https://uptimeinstitute.com/
  • The Open19 / OCP power & busbar design notes — for the power-shelf/busbar model
  • Barroso, Hölzle, Ranganathan, The Datacenter as a Computer, 3rd ed. — the canonical systems view of warehouse-scale machines (free online)
  • Qualcomm Cloud AI — product context for the accelerators this role's racks carry — https://www.qualcomm.com/products/technology/processors/cloud-artificial-intelligence
  • Cross-track: Head of Software Engineering — GPU, Phase 01 WARMUP — the silicon inside the accelerators you manage