The Role — Frontier Pre-Training Lead

Modelled on: Vlad Feinberg — Distinguished Engineer, Google DeepMind; area lead for Gemini pre-training (Flash / Flash-Lite). Sources: his April 2025 Princeton talk "Gemini Pretraining" (slides), his personal site, and his interview on Developing Dev — "Google DeepMind Pre-Training Lead: How To Get a Job at a Frontier Lab" (video).

This document is not a copy of a job posting. It is a reconstruction of what the job actually is — the scope, the decisions, the failure modes, the artifacts — assembled from what he has said publicly about doing it. Everything in the rest of this track exists to make you able to do the things listed here.

Table of Contents

• 1. One-Paragraph Summary of the Job
• 2. Career Timeline — What the Path Actually Looked Like
• 3. What "Pre-Training Area Lead" Owns
• 4. The Three Research Verticals
• 5. The Products That Depend On You
• 6. Synthesized Job Description
• 7. The Skill Matrix, Graded
• 8. How He Says To Get Hired
• 9. What This Track Does About It

1. One-Paragraph Summary of the Job

You decide how to convert a fixed pile of compute into the best possible model — and "best" is not test loss, it is quality per served token per watt. You own the recipe (architecture, data mixture, optimizer, schedule), the forecast (a scaling law that says what loss you will land on before you spend $10M of TPU time), the compression pipeline that turns a giant teacher into a small servable student, and — when the run is live — the pager. You are simultaneously a researcher (navigating ideas that may not work), a distributed-systems engineer (a training job across many datacenters), an economist (power is the bill), and an SRE (the run must not die). Feinberg's framing of the two halves is exact: software engineering is a deterministic DAG where you "just make monotone progress"; research is a stochastic one, where "some of the nodes... may or may not work out."

2. Career Timeline — What the Path Actually Looked Like

Taken from the "About Me" slide of his own talk. Study the shape, not the brand names — the shape is the lesson.

Four things to extract from that table

1. The through-line is "make the expensive thing cheap." Distributed lasso for DB cubing → quantizing an ads DNN → inference-efficient LLMs → Flash. That is one skill, re-applied at four scales. It is not a career of unrelated hops.
2. The unglamorous job was the on-ramp. He describes doing SFT for an early version of Bard — "hyperparameter tuning and engineering work to get this model running on some really old TPUs" — and getting a spot bonus from Jeff Dean for it. He had been in pure research optimizing for first-author papers at NeurIPS/ICML/ICLR; his manager Rohan Anil pushed him toward the product-shaped work. That pivot is the whole story.
3. He did not finish the PhD. The credential is not the gate. What the PhD teaches — research taste, literature traversal, mathematical maturity — is the gate. This track teaches those directly.
4. Ads pCTR quantization is the unsexy ancestor of Flash. His own advice: "chase the problems that people are facing in the world today... don't be afraid to tackle a smaller part of this problem or maybe a more menial sounding part."

3. What "Pre-Training Area Lead" Owns

Think of it as five distinct hats worn by the same person in the same week.

Hat 1 — The recipe owner

A recipe is the full parameterized specification of a training run: architecture family (dense vs MoE, depth/width ratio, attention shape), tokenizer, data mixture and its curriculum, optimizer and its hyperparameters, learning-rate schedule, batch-size schedule, numerics (bf16/fp8, what stays in fp32), and the parallelism plan. You do not tune these per-run — you tune the function that maps compute budget C to all of the above, because you only get to run the big one once.

Hat 2 — The forecaster

Before the flagship run, you must be able to state the final test loss with error bars. His framing of why this is the problem:

Old ML: iterate on CIFAR-10, then apply to ImageNet. LR searches by doing multiple "final runs" — "the last data point is our test set!"

Now: "every single time you go up for a pre-training run, you're about to put in more FLOPs into this run than you've ever done before." Every next run requires extrapolation.

So you run a ladder of small models, fit a law, and extrapolate. Phase 01 and Phase 02 are this, end to end.

Hat 3 — The inference co-designer

Chinchilla-optimal is the wrong objective for a model that will serve billions of tokens. You trade training FLOPs against serving FLOPs, and you choose architecture shapes that saturate the specific hardware units you will serve on. His talk's napkin math is the canonical example (Phase 05 rebuilds it exactly).

Hat 4 — The compressor

Distillation, quantization, and serving-friendly architecture changes. He calls the fact that you can drop weights from FP32 down to 4 bits and keep quality "kind of a miracle" — and the reason it matters is economic: "99% of the total cost of operation for AI hardware comes from the power that it takes to run these chips."

Hat 5 — The SRE

When the flagship run is live, someone has to keep it alive. For Flash 2.0 that was 40 days of a five-person rotation, handing off day by day across Paris and Mountain View, watching data iterators, chasing indexing bugs, avoiding "wasting all of this GPU time." He is blunt: "We did not do a lot of sleeping."

4. The Three Research Verticals

His team's stated research surface. Each one gets its own phase in this track.

5. The Products That Depend On You

From his talk's "Small Model Customers" section — the reason Flash and Flash-Lite exist is that Google's biggest surfaces need (1) high volume and (2) real time:

• Free-tier Gemini app (the chatbot)
• AIO — AI Overviews in the search bar
• AIM — AI Mode in the search bar
• Vertex AI — customer fine-tuning and deployment
• AI Studio — the generation API
• Astra (real-time multimodal assistant) and Mariner (web-interaction agent)
• Plus models for Ads and YouTube, and technical work on the Google–Apple partnership

And the internal cadence he names: the Gemini "tick-tock" — this generation's Flash should match last generation's Pro. That single sentence is a compression target, a distillation target, and a scaling-law target all at once.

6. Synthesized Job Description

What the posting would say if it were honest.

Title: Research Engineer / Research Scientist — Pre-Training (Frontier LLM)

You will:

• Own a parameterized pre-training recipe: architecture, data mixture, optimizer, schedule, numerics, and parallelism plan, as a function of the compute budget.
• Design and run scaling-law ladders (IsoFLOPs and/or parametric L(N, D) fits) and publish forecasts with uncertainty for flagship runs, including the decision rule for baseline-vs-candidate recipe comparison.
• Extend scaling laws along non-classical axes: unique tokens and repeat count L(N, U, R), routed/MoE capacity, inference-aware total-cost objectives, and distillation (teacher-size / teacher-compute allocation).
• Co-design architectures for the serving target: choose d_model, d_ff, head count/shape, KV-head sharing, and expert count against a roofline model of the target accelerator; defend the choice with MFU and latency arithmetic, not vibes.
• Build and operate the compression pipeline: teacher-logit generation at trillion-token scale (storage format, multi-datacenter data movement), distillation objectives, and post-training / quantization-aware paths down to 4-bit.
• Partner with serving to make architectures deployable — expert parallelism, pipelined prefill, prefill/decode disaggregation, KV-cache budgets.
• Keep flagship runs alive: goodput, checkpoint/restart, loss-spike diagnosis and mitigation, data-iterator correctness, silent-data-corruption detection, on-call rotation.
• Make and defend go/no-go calls under uncertainty; run "a very transparent technical process" when a large bet (e.g. dense → MoE) is on the table.

You must be able to:

• Derive C ≈ 6ND and the full per-step FLOP count of a transformer from the layer shapes, by hand, and say exactly where the approximation breaks (attention, MoE, embeddings).
• Read the primary literature and traverse a citation tree to find "the high-value papers."
• Implement a technique from a paper yourself, efficiently — including at the kernel level.
• Reason about collectives (all-reduce/all-gather/reduce-scatter/all-to-all) and their cost in bytes and in latency, and about how a sharding decision changes convergence.
• Explain why an MFU number is what it is, unit by unit.

Signals that get you the interview (his words, verbatim in spirit): "intent, mathematical maturity, grit."

7. The Skill Matrix, Graded

Grade yourself 0–3 (0 = never heard of it, 3 = I have shipped this). The right column is where this track fixes the gap.

8. How He Says To Get Hired

He is unusually concrete about this, so it is worth listing exactly.

1. Produce "actual evidence that you've created something of use to other people." Contribute to vLLM, SGLang, TensorRT — real improvements to real serving stacks. He calls this "an extremely positive signal."
2. Do the exercises in How To Scale Your Model ("The Scaling Book") — handwritten, detailed — and record yourself doing them. He publicly offered interviews (and referrals when he had no headcount) to people who did this plus the transformer exercise.
3. Implement a transformer from scratch. It demonstrates "willingness to get into the weeds engineering-wise" and fluency in the "bread-and-butter math that we use every day to size these LLMs."
4. Be able to traverse a citation tree. Knowing which papers are the high-value ones for a topic is itself the skill.
5. If you are already inside a big company: don't try to teleport. Become the person who owns LLM adoption for your product area. You then become "the partner that we work with on the research side" — and the transfer becomes a formality. (He names Nate Lintz, who transferred in and now owns much of their inference work.)
6. Be someone people want to see succeed. His anti-Machiavelli argument: build projects that leverage other people's complementary skills so they shine, and they will show up for your next project. He credits Todd Lipkin with first getting him into CS.

On the doom: he calls the discourse "FUD everywhere." His counter is accountability — "you can't hand off blame to AI." A lawyer can't be replaced by a model that "can't be disbarred." Someone must be responsible for the output and for how resources get allocated. His practical version: "we all have agency over our future and we can start investing in skills that matter for tomorrow today."

9. What This Track Does About It

Every phase is: a README.md (why the phase exists + lab spec), a WARMUP.md (zero-background → principal-level explanation of every term, with mechanism, math, code, diagrams, misconceptions, tips and takeaways), and at least one runnable lab — pure Python stdlib, offline, deterministic, test-verified — that builds the mechanism itself rather than calling a framework.

You will finish having personally implemented: the FLOP counter, the IsoFLOPs fitter, the L(N,U,R) and inference-aware optimizers, a top-k MoE router with capacity and auxiliary losses, a collective-cost simulator, a roofline/MFU model, a pipelined-prefill scheduler, a distillation loss with temperature and a teacher-logit store, a 4-bit quantizer with a power model, a training-stability watchdog, a PPO step, a tile-DSL interpreter, an MDP research planner, and a capstone that chains all of it into one defensible technical memo.

Start here: Transcript, Dissected — every claim he made, unpacked for someone with no background — then Phase 00.

References

• Vlad Feinberg, Gemini Pretraining, Princeton, Apr 2025 — slides (PDF)
• Vlad Feinberg, personal site and about
• Ryan Peterman, Developing Dev: Google DeepMind Pre-Training Lead: How To Get a Job at a Frontier Lab · video
• Kaplan et al., Scaling Laws for Neural Language Models, 2020 — https://arxiv.org/abs/2001.08361
• Hoffmann et al., Training Compute-Optimal Large Language Models (Chinchilla), 2022 — https://arxiv.org/abs/2203.15556
• Sardana et al., Beyond Chinchilla-Optimal, 2024 — https://arxiv.org/abs/2401.00448
• Muennighoff et al., Scaling Data-Constrained Language Models, 2023 — https://arxiv.org/abs/2305.16264
• Busbridge et al., Distillation Scaling Laws, 2025 — https://arxiv.org/abs/2502.08606
• Clark et al., Unified Scaling Laws for Routed Language Models, 2022 — https://arxiv.org/abs/2202.01169
• Pope et al., Efficiently Scaling Transformer Inference, 2022 — https://arxiv.org/abs/2211.05102
• Austin et al., How To Scale Your Model — https://jax-ml.github.io/scaling-book/
• Xu et al., GSPMD, 2021 — https://arxiv.org/abs/2105.04663
• DeepSeek-AI et al., DeepSeek-V3 Technical Report, 2024 — https://arxiv.org/abs/2412.19437
• Jacob Steinhardt, Research as a Stochastic Decision Process — https://cs.stanford.edu/~jsteinhardt/ResearchasaStochasticDecisionProcess.html