👨🏻 Brother Talk — Phase 05, Off the Record

ML compilers & IR — ONNX, TVM, MLIR. The JD lists "ML compiler workload synthesis" as a plus and "hardware software co-design" as preferred. This phase is that, made concrete.

Brother, here's the big picture nobody draws for you clearly: between your PyTorch model and the transistors on a Qualcomm SOC, there is a stack of intermediate representations, and the whole game of model deployment is lowering cleanly through them. PyTorch → FX/aten (P04) → ONNX → a hardware compiler IR (TVM's Relay/TIR, MLIR dialects) → the NPU's instruction set. Each level throws away some flexibility and gains some optimizability. The Senior Staff engineer is the person who understands the whole pipe and can debug a model that lowers fine until level 4 and then explodes. That's a rare and valuable shape of knowledge.

Start with the truth about ONNX, because it's the universal interchange format and the place most deployment bugs surface: ONNX is a contract between frameworks. You export from PyTorch, and the export either captures your model faithfully or it doesn't — and "doesn't" is common. Unsupported ops, dynamic shapes that become fixed, control flow that doesn't survive, opset version mismatches. The export lab will teach you that the export is where the accuracy bug is born: a model that's perfect in PyTorch can drift after ONNX export because an op got decomposed differently or a shape got specialized. The discipline — export, then verify numerically against the original, then optimize — is the same correctness-first discipline as P04, and it's the one that saves you. (And note: the dynamo-based exporter needs onnxscript, optimization needs onnxoptimizer — these are real deps; on a bare machine the lab tells you what's missing rather than lying.)

The mental model for IR design that'll serve you forever: an IR is a deliberate tradeoff between expressiveness and optimizability. High-level IR (ONNX, Relay) keeps operators coarse (a whole "conv" or "attention") — easy to pattern-match and rewrite, but coarse. Low-level IR (TIR, LLVM) is fine-grained loops and memory — you can optimize the daylights out of it, but you've lost the high-level structure. Lowering is the act of going from coarse to fine, and every lowering step is a chance to optimize and a chance to introduce a bug. MLIR's whole thesis is "don't pick one level — have many dialects and progressively lower through them." When you get the MLIR lab working (Python bindings), you're touching the framework that modern hardware compilers — including a lot of Qualcomm's world — are built on.

Here's the Qualcomm-specific intuition: operator coverage is destiny. Your fancy model is only as deployable as the least-supported op in it. One unsupported op on the NPU means a CPU fallback, which means a round-trip off the accelerator, which means latency death. So a huge part of this job is workload synthesis — understanding which operators the hardware supports, and either (a) rewriting the model to use supported ops, or (b) decomposing the unsupported op into supported ones, or (c) escalating to the hardware team that this op needs a kernel. The compiler is where you discover and fix the coverage gaps. That's the "co-design" the JD wants: you're not just consuming the hardware, you're feeding back what it needs to support.

TVM is worth your respect even though it may skip on this machine (it needs the TVM runtime). Its auto-tuning idea — search the space of loop tilings/schedules to find the fastest kernel for this hardware — is the conceptual ancestor of a lot of modern compiler optimization. You don't need to run it to understand the lesson: the optimal kernel schedule is hardware-specific and often non-obvious, so let a search find it. That mindset (measure, don't guess; search the space) is the bridge to the profiling phase (P07).

The thing nobody tells you about compiler work: it's debugging, not building. You'll spend 90% of your time figuring out why a lowering went wrong — why this op didn't fuse, why this shape got specialized, why the numerics drifted at this level — and 10% writing the pass. So the skill that matters is reading IR and bisecting the pipeline: dump the IR at each level, find the level where it first goes wrong, and you've localized the bug. The person who can read an ONNX graph or an MLIR dump and say "ah, the problem is here" is the person who unblocks the whole deployment.

Career truth, brother: ML-compiler skills are a genuine moat. The pool of people who understand PyTorch internals (P01/P04) and the compiler/IR stack and the hardware target is tiny — and that intersection is exactly the Qualcomm Senior Staff role. Most ML people stop at the framework; most systems people don't know the ML. You're building the rare combination. Lean into the "lowering through IRs" mental model — it's the spine that connects everything from torch.compile to the NPU.

Build the ONNX export, verify it numerically, optimize it. Read an MLIR dump. Then come to P06, where the IR finally meets the metal — the Qualcomm NPU itself.

— your brother 👨🏻