🛸 Hitchhiker's Guide — Phase 09: Hardware Abstraction (The Keystone)

Read this if: you've heard "decouple software from hardware" and want to know what it mechanically means. This is the JD's thesis. Field notes; derivations in the WARMUP.

0. The 30-second mental model

A HAL = stable API (what apps call) + backend ABI (a vtable of function pointers + capability struct that vendor plugins implement) + dlopen (load backends at runtime; core links none of them). Beat lowest-common-denominator with capability negotiation. Keep the ABI loadable across years with version-first, append-only, reserved fields. Make it trustworthy with a conformance suite. This layer is the platform's moat (Ch. 9).

1. API vs ABI (the thing everyone blurs)

• API = source contract (names, signatures). Breaking it breaks compiles.
• ABI = binary contract (struct layout, offsets, calling convention, symbol names, enum values). Breaking it breaks already-built binaries — silently.
• Backends are separately-compiled .so files sharing only the ABI with your core. That's why ABI discipline is the whole game.

ABI rules: version field FIRST · fixed-width types (int32_t) · append-only struct growth · reserved[N] fields · one exported symbol · enums with explicit values.

2. The pattern (Lab 01 in one diagram)

app -> stable API -> HAL core -> vtable dispatch -> backend.so (dlopen'd)
                                  struct Backend { abi_version; caps; (*matmul); ... }
cpu_ref.so   fast.so   limited.so   <- swap/add WITHOUT recompiling the core

Same pattern as CUDA driver/cubins, OpenCL ICDs, LLVM backends, Postgres extensions. Universal mechanism; the discipline is what's rare.

3. Capability negotiation (beat the LCD)

Each backend declares dtypes/ops/features. Platform queries → routes work to a capable backend → exposes the union, fails cleanly on the unsupported. Don't expose only the intersection of all backends — that taxes good hardware for the worst backend's limits. (Same shape as Phase 06 device-plugin advertising and Phase 07 engine routing.)

4. Abstraction vs performance (the hard part)

Too abstract = slow/leaky; too concrete = vendor-coupled. Four fixes:

1. Escape hatches (generic path + optional native fast path)
2. Operation-level ops (attention, not matmul+softmax) ← the winner
3. Compiler layer (Triton/MLIR lowering) ← highest ceiling, highest cost
4. Capability-gated fast paths

Default: operation-level + capability-gated fast paths + escape hatch. Abstract at the right altitude.

5. Versioning so vendors don't hate you

version-first check · append-only growth · reserved fields · flags over fields · semver (major=breaking, minor=additive), core accepts a range. Break vendor backends every quarter → no ecosystem → no product. The ecosystem IS the product.

6. Conformance (what makes it trustworthy)

One test matrix, all backends pass: correctness (with cross-vendor rounding tolerance — Phase 03), declared-capability verification, and (advanced) a performance tier (X% of roofline or "supported, not recommended"). "Certified backend" status is both a quality promise and an OEM-negotiation lever.

7. Backend strategy (Phase 03 Ch. 9 from the builder's seat)

Blend: vendor backends now + Triton/MLIR for velocity, all behind YOUR HAL. Budget for the AMD asymmetry (no PTX-equivalent; per-gfx rebuilds — Phase 03).

8. The business case (why a Head of Eng owns this)

• OEM integrations = backends behind your HAL = direct revenue lever (the JD's commercialization mandate).
• It inverts NVIDIA's software moat — sells lock-in freedom to customers and market access to non-NVIDIA chips.
• Conformance certification = a commercial gate you control.
• A performant, versioned, conformance-gated multi-vendor HAL is years of moat, not an afternoon's wrapper.

9. War stories

• "Our HAL ran everywhere at 25% of peak" — abstracted at the instruction level (generic matmul), couldn't reach Tensor Cores. Re-abstracted at op level (attention) with capability-gated fast paths. Peak recovered.
• "A vendor's old .so corrupted memory after our release" — we inserted a struct field in the middle; their year-old binary read every later field at the wrong offset. Now: append-only + reserved fields + version gate. Never again.
• "Backend declared FP16, produced garbage" — declarations were trusted, not tested. Conformance suite now verifies every declared capability.
• "We shipped one engine, lost a deal needing AMD" — no HAL, NVIDIA-coupled. The HAL is exactly the thing that would have made that deal a config change.

10. Exit bar

You can design and build a versioned, dlopen-based HAL with capability routing and conformance; explain API vs ABI and ABI evolution cold; pick the portability-posture blend; and argue the moat. You've added a backend without recompiling the core. This is the JD's thesis, in your hands. Next: Phase 10 — making all of it observable and operable in production.