Brother Talk — Foundations

Off the record. The stuff I'd tell you over a beer, not in a design review.

This phase is the one that makes people respect you, and almost nobody does it. Everyone can prompt a model. Very few engineers — even ones with "AI" in their title — can tell you, on the spot, how many GPUs it takes to serve a 34B model at 1k QPS and why. The moment you can do the FLOPs/memory/KV-cache/cost math out loud, you stop being "the person who calls the API" and become "the person we ask before we commit budget." That reputation is worth more than any framework you'll learn later.

The KV-cache is the thing that will save you at 2 a.m. I promise you that at some point a service will OOM in production and a roomful of smart people will be staring at it confused because "the model fits, we checked." You'll be the one who says "what's the concurrency × context? That's the KV-cache, and it's bigger than the weights." You'll fix it in ten minutes. Memorize the KV-cache formula like a phone number.

"Memory-bound decode" is a cheat code for sounding senior. Half the inference-optimization conversations in this industry are people reinventing the same insight: decode re-reads the weights every token, so it's bandwidth-bound, so batch and quantize. Once you really get the roofline, you'll watch teams waste months and money throwing FLOPs at a bandwidth problem, and you'll be able to redirect them in one sentence. That one sentence is a promotion.

Resist the bigger-model dopamine. There is a real, almost emotional pull toward "use the biggest, smartest model." It feels safe and impressive. But you pay for inference on every single request for the life of the product, and that cost compounds in a way training cost doesn't. The senior move — the one that's boring in the demo and heroic on the P&L — is the smallest model that clears the bar, quantized and distilled. Train yourself to feel good about shipping small.

Numbers end arguments; opinions start them. When you walk into a design review with "here's the FLOPs, here's the KV-cache at peak, here's the $/1M, and the deciding dial is cost," the conversation is over in your favor. When you walk in with "I think we should use the smaller model," you're in for an hour of vibes. Do the arithmetic before the meeting. Every time.

You don't need a PhD to do this, but you do need to not be scared of the math. The math in this phase is multiplication and one min(). That's it. The reason it's a differentiator isn't that it's hard — it's that most people never bother to make it a reflex. Be the person who bothered.

What's actually worth caring about: the four numbers (FLOPs, weights, KV-cache, $/1M) and the deciding dial. What's not worth losing sleep over: the exact constant in the scaling-law fit, or whether it's 6ND or 6.02ND. Estimates that are right to within 2× win design reviews; false-precision to 5 decimals just slows you down.

The career framing. This phase is the foundation of the "I own the model stack" identity. Every later phase — attention, quantization, serving, agents — is more impressive, but this is the one that makes the rest credible, because it proves you think about models as systems with budgets, not as magic. Get this cold and the whole track compounds. Now go make pytest green.