👨🏻 Brother Talk — Phase 02, Off the Record

Architecture mastery — where the JD says "expertise in model architecture, backed by literature rationale and intuition." That word intuition is the whole game.

Brother, read the JD line again: "Expertise in model architecture, backed by literature rationale and intuition." Most candidates hear "know the architectures" and go memorize that a transformer has attention and an FFN. That is not what this asks. It asks for rationale — why this design and not another — and intuition — what happens to accuracy and latency when you change it. At Senior Staff level you're not implementing someone else's paper; you're the person who looks at a new architecture and says "this MoE will be a nightmare on our NPU because of the gather/scatter, but here's the variant that won't." That judgment is what this phase builds.

Here's the reframe: architectures are not a zoo of unrelated animals; they're a sequence of answers to a small set of recurring problems. Attention is quadratic → so we get sparse attention, linear attention, and eventually SSMs (Mamba) that ditch attention for a recurrence with near-linear cost. Dense FFNs are expensive → so we get Mixture-of-Experts that activates only k of N experts. Vision needs inductive bias → ViT throws it away and recovers it with scale and self-distillation (DINO). Once you see architectures as moves in response to a constraint, you can reason about ones you've never seen — which is exactly the interview test.

Now the part that matters for this job specifically, and that pure-ML people miss: the best architecture on a GPU benchmark is often the worst architecture on an NPU. This is the Qualcomm lens, and it's your edge. MoE looks great on FLOPs-per-token, but the expert routing is a dynamic gather/scatter that NPUs (which love static, dense, predictable dataflow) hate — you get memory-bound stalls and CPU fallbacks. SSMs have a beautiful linear recurrence, but the scan is sequential and maps awkwardly to hardware built for parallel matmuls. A candidate who can say "yes the paper shows X, but on a mobile NPU the real bottleneck is the irregular memory access, so here's what I'd actually ship" — that person gets hired, because that's the literal job: adapt GenAI techniques for Qualcomm SOCs.

The MoE lab will teach you the thing that trips everyone: load balancing. Naive top-k routing collapses — a few experts get all the tokens, the rest die, and your "8 experts" is really 2. The auxiliary load-balancing loss isn't a footnote; it's what makes MoE work at all. And on hardware, an unbalanced MoE is a latency disaster because the busiest expert gates the whole layer. Build it, watch the collapse happen, watch the aux loss fix it. That's intuition you can't get from reading.

The SSM/Mamba lab is the one that'll make you feel smart, because the selective-scan recurrence is genuinely elegant — and it's the future-facing skill (the JD wants "latest GenAI techniques adapted for NPUs"). But hold the Qualcomm skepticism: the parallel scan that makes Mamba fast on a GPU is exactly the part that's hard to map to an NPU's dataflow. Knowing both the elegance and the hardware friction is the Senior Staff take.

On ViT + DINO (the lab I had to fix the test for — the loss genuinely decreases, the old test just used an unfair baseline): the lesson beyond the architecture is self-distillation without labels. A teacher that's an EMA of the student, centering to prevent collapse, sharpening to make confident targets — it's a deceptively deep training dynamic. The collapse modes (everything maps to one vector; everything maps to uniform) are real failure modes you'll see in production self-supervised training, and centering vs sharpening are the two knobs that fight them. Understanding why it doesn't collapse is more valuable than the architecture itself.

Multimodal (CLIP→LLM bridge): this is where the field is going and where edge GenAI gets interesting. The bridge — projecting vision features into the LLM's token space — is a small module with outsized importance, and on-device it's where a lot of the accuracy/latency tradeoff lives. Know how the modalities get glued, because "run a multimodal model on a phone" is a sentence you'll hear at this job.

Career truth, brother: architecture knowledge ages, but architecture intuition compounds. The specific models will be obsolete in two years; the ability to look at a new one and reason about its accuracy, its cost, and its hardware-friendliness is permanent. In interviews, the tell of a senior person is that they don't just describe an architecture — they critique it, they know its failure modes, and (for this role) they know how it'll behave on silicon. Read the papers for the rationale, build the labs for the intuition, and always ask "how does this map to the NPU?" — that question is your whole differentiator.

Build the variants. Make the MoE collapse and then fix it. Then come to P03, where we make these architectures small enough to ship — quantization, where accuracy goes to die if you're not careful.

— your brother 👨🏻