Interview Prep — Inference Optimization & Solutions Architect (JD2)

This is the rehearsal layer. The knowledge modules build the depth; this folder makes you able to defend it out loud, fast, under pressure — which is what actually gets you hired into JD2. A solutions-architect interview tests three things, and there's a file for each:

FileTestsFormat
01 — Concepts Rapid-FireDo you actually understand the systems?60+ Q&A, whiteboard answers
02 — System DesignCan you size and architect a deployment?5 full walkthroughs
03 — Behavioral & CustomerAre you a trusted advisor, not just a coder?STAR stories + customer role-plays

How JD2 interviews are structured (what to expect)

A 10–15-YOE accelerator-solutions role typically runs:

  1. Phone screen — résumé + "explain prefill vs decode" / "what's the KV cache" sanity checks. File 01 covers this cold.
  2. Deep technical — quantization, parallelism, profiling, a debugging scenario. Files 01 + the knowledge modules.
  3. System design — "size and design inference for [workload]." File 02 — the highest-weight round.
  4. Customer/whiteboard role-play — they play a customer with a vague/contradictory ask; you run discovery, size it, and present trade-offs. File 03 — unique to solutions roles, where most engineers are weakest.
  5. Behavioral/leadership — escalations, cross-team conflict, a project you owned end-to-end. File 03.

The differentiator at the principal level isn't knowing the most facts — it's fluently moving between the silicon and the customer's spreadsheet, and turning a vague ask into a measurable spec and a trade-off curve. Every file drills that move.

The 10 answers you must have flawless

If you can deliver these crisply, you're 80% there. Each links to the depth.

  1. Prefill vs decode — why one is compute-bound and the other memory-bound, and why that changes everything. (K01 §4)
  2. The roofline — drawn on a whiteboard, with the ridge-point arithmetic, used to diagnose a workload. (K00 §5)
  3. KV cache — what, why, the size formula, and why it caps concurrency. (K01 §3)
  4. Why naive INT8 breaks LLMs — activation outliers, and the three fixes. (K02 §8)
  5. Continuous batching + PagedAttention — the two ideas that make serving economical. (K04 §3–4)
  6. Tensor vs pipeline parallelism and why TP wants NVLink. (K05 §3–4, §8)
  7. "The GPU is idle vs busy" profiling fork and what each implies. (K06 §4)
  8. FFmpeg vs GStreamer and decode-bound video sizing. (K07 §3–4)
  9. Sizing math — SLO → #accelerators → $/token. (K08 §2–3)
  10. Model rightsizing — recommending down when justified, and why that builds trust. (K08 §5)

Rehearse each as a 60–90 second spoken answer with a drawing. Record yourself. If you stumble, the gap is in the linked module — go back, then re-rehearse.