Track G — Agentic Coding
Reported as a beta fifth round, not administered to everyone: an existing codebase plus a problem too large to hand-write in the time, worked through using an AI coding agent (
../../research/source-report.mdrows 37–40).The research says do not skip this. The source report frames it as an optional beta, which invites skipping. Independent corroboration says the format is going industry-wide.
→ Study guide: WARMUP.md — the method in eight steps, a fully worked 60-minute transcript with scoring commentary, and five diffs to accept or reject.
→ DIFFBANK.md — 30 agent diffs, 90 seconds each. The round is scored on which of the agent's output you let through, and that is a trainable skill: the six-pass review, the ranked taxonomy of what agents actually get wrong, and three diffs that look wrong and are right (because rejecting everything is also a failure).
Table of Contents
- Why This Is Not Optional
- Ask the Recruiter First
- What Is Actually Being Scored
- The Environment
- The Six Tasks
- The Method
- The Good-vs-Bad Rubric
- Drill Set
- Failure Modes
- Self-Assessment Rubric
- References
Why This Is Not Optional
Corroborated independently of the source report
(../../research/findings.md):
Meta began rolling out AI-enabled coding interviews in late 2025; Google is piloting a
Gemini-assisted coding format; CodeSignal shipped agentic coding assessments as a product.
The reported common shape: 60 minutes, a multi-file codebase, phased objectives — bug fix, then core implementation, then optimization — with the assistant confined to a chat panel rather than given direct file-edit authority. Stated evaluation criteria converge on AI fluency: prompt construction, output validation, and debugging the assistant's work.
Note that the phased structure is the same progressive-gate format as Track A. The gating skill transfers directly; what is new is driving a second agent through it while narrating your strategy.
Ask the Recruiter First
Policies at companies you may interview at in the same month are opposite.
| Company | Reported policy |
|---|---|
| OpenAI | Agentic round in beta, some candidates |
| Meta, Google | Actively piloting AI-assisted rounds |
| Anthropic | Reportedly prohibits AI tools in live interviews; candidates reportedly removed for using them |
So: ask, per company, per round. Never assume, in either direction. This goes on the
pre-interview checklist in ../../STATE.md, and it is not a question that
makes you look unprepared — it makes you look like someone who reads the rules.
What Is Actually Being Scored
Not "can you use an AI." Everyone can. The scored skill is engineering judgement applied to a fast, confident, occasionally wrong collaborator — which is a real and specific skill.
| Scored | Not scored |
|---|---|
| Decomposing an oversized problem into verifiable units | Typing speed |
| Writing a plan before invoking the agent | Prompt "tricks" |
| Giving the agent checkpoints you can actually verify | Volume of code produced |
| Reading diffs critically and rejecting bad ones | Accepting everything that compiles |
| Keeping tests green throughout | Green at the end only |
| Knowing when to take over manually | Delegating everything on principle |
| Narrating the strategy out loud | Silent driving |
The tell that separates the top quartile: the candidate who rejects an agent's output and says why. Someone who accepts every diff is not reviewing, and the interviewer cannot distinguish them from someone who does not understand the code. Rejecting one plausible-looking diff with a specific reason is worth more than three accepted ones.
The Environment
cd tracks/agentic
# Clone a mid-size, well-tested pure-Python repo with a real test suite.
# Good properties: 10k-60k LOC, fast tests, no heavy native deps, active history.
git clone --depth 50 <target-repo> workspace
cd workspace && python3 -m pytest -x -q # establish the green baseline FIRST
Suggested targets — pick one you have not worked in, because reading unfamiliar code under time pressure is half the round:
| Repo | Why it works |
|---|---|
httpx | Async + sync dual API, clean layering, fast suite |
pydantic (v1 branch) | Type machinery, lots of edge cases |
rich | Rendering pipeline, plenty of pure-logic surface |
sqlglot | Parser/transpiler — dense, well-tested, great for refactors |
flask / click | Small, canonical, good for cross-cutting changes |
Record the baseline: test count, runtime, and coverage if available. Every task below is scored partly on whether that baseline stayed green the entire time, not just at the end.
The Six Tasks
Each is deliberately too large to hand-write in 60 minutes. That is the point — the constraint is what forces delegation, and delegation is what is being scored.
| # | Task | Shape | Skill it isolates |
|---|---|---|---|
| G1 | Cross-cutting refactor — change a signature or pattern used in 30+ call sites, preserving behavior | Wide, shallow | Batching mechanical work; verifying breadth you cannot read |
| G2 | Feature spanning modules — add a capability touching parsing, core logic, and public API | Deep, narrow | Decomposition; sequencing; keeping the suite green mid-flight |
| G3 | Performance fix requiring profiling — find the hot path, fix it, prove the improvement | Investigative | Making the agent measure rather than guess |
| G4 | Test-coverage expansion — take an under-tested module from 40% to 85% with tests that would actually catch bugs | Generative | Rejecting assertion-free tests; the agent's default failure mode |
| G5 | Bug in unfamiliar code — a real reverted commit, re-applied. Find and fix it from a failing test | Diagnostic | Directing an investigation, not a code generation |
| G6 | Migration — move a subsystem to a new API or library, with a deprecation path | Broad + risky | Planning; incremental verification; knowing when to take over |
Each gets a directory holding: your written plan, the transcript, the final diff, the test log, and your self-score.
The Method
This is the thing to internalize. It is the difference between driving and hoping.
1. Establish the baseline before touching anything
Run the tests. Record the count and the runtime. You cannot claim you kept it green if you never knew it was green.
2. Read enough to write a plan
Ten minutes, maximum, of reading. Enough to name the files that will change and the invariant that must hold. Not enough to understand everything — you will not have time, and the round knows that.
3. Write the plan before invoking the agent
In the chat, out loud, or in a scratch file. It must contain:
- The end state, in one sentence
- The checkpoints — three to five, each independently verifiable
- The invariant that must hold throughout (usually: this test file keeps passing)
- What you will do manually rather than delegate
Narrate this. "I'm going to do the mechanical rename with the agent because it's 30 files and I can verify it with the test suite, but I'm writing the state-machine change myself because the invariant is subtle and I want to be the one who understands it." That sentence alone is a large share of the score.
4. Delegate in checkpoint-sized units
Not "implement the feature." Not "fix line 42." One verifiable unit: "Add the timeout
parameter to these four public functions, thread it through to the transport layer, and keep
tests/test_timeout.py passing. Do not change the default behavior."
5. Verify every checkpoint before proceeding
Run the tests. Read the diff. Read the diff even when the tests pass — a passing suite proves you did not break what was tested, not that you built what you meant.
6. Reject bad work, specifically
"This adds a bare except Exception in the retry path, which will swallow
CancelledError and make the client uncancellable. Use except httpx.TransportError
instead." Specific, technical, cites the consequence. That is a review, and it is what the
round is for.
7. Take over when the agent is looping
Two failed attempts at the same checkpoint means the problem is under-specified or genuinely subtle. Write it yourself and say why: "It's fighting me on this because the constraint isn't expressible in the test — I'll do it directly." Recognizing that boundary is a senior signal; grinding through a third attempt is not.
8. Narrate continuously
Same rule as Track A. Silence reads as lost. Say what you are delegating and why, what you are checking, and what you rejected.
The Good-vs-Bad Rubric
| Dimension | Someone who pastes prompts and hopes | Someone driving |
|---|---|---|
| Before starting | Types the task into the agent | Runs tests, reads, writes a plan with checkpoints |
| Unit of delegation | The whole problem | One verifiable checkpoint |
| Prompting | "Make it work" | States the invariant, the boundary, and what not to change |
| On output | Accepts if it compiles | Reads the diff; runs tests; questions anything unexplained |
| On failure | Re-prompts with "that didn't work" | Diagnoses why, then re-scopes or takes over |
| Tests | Green at the end, maybe | Green at every checkpoint |
| Test quality | Accepts whatever the agent wrote | Rejects tests that assert nothing meaningful |
| Manual work | Never, on principle | Takes the subtle parts personally |
| Narration | Silent | Continuous strategy commentary |
| At time | A large diff, unverified | A smaller diff, verified, with a stated plan for the rest |
A smaller verified diff beats a larger unverified one. That is the whole rubric in one line, and it maps directly onto Track A's "get something correct early" — an unverified change is an unopened gate.
Drill Set
| Drill | Cadence | Trains |
|---|---|---|
| Timed task run | Weekly, 60 min | The round. One task, recorded, scored |
| Plan-only, 10 min | 3×/week | Read a task, write the checkpoint plan. No agent |
| Diff review | 2×/week | Take an agent diff and find three things wrong with it |
| Rejection practice | Weekly | Deliberately accept a bad diff, run tests, find what broke. Calibrates trust |
| Takeover judgement | Weekly | Note every point you considered taking over. Review whether you were right |
| Narration replay | Weekly | Listen back. Was the strategy audible? |
| Cold repo | Monthly | A repo you have never opened. Reading speed under pressure is the hidden variable |
Failure Modes
| Failure | Symptom | Fix |
|---|---|---|
| No plan | Straight to prompting | Plan-only drill |
| Delegating the whole problem | One giant prompt, one giant diff | Checkpoint-sized units |
| Not reading diffs | Accepts anything green | Diff-review drill |
| Accepting empty tests | Coverage up, quality flat | G4 specifically; assert on behavior, not on calls |
| Never taking over | Third attempt at the same checkpoint | Two-strike rule |
| Taking over too early | Hand-writing the mechanical 30-file rename | That is exactly what to delegate |
| Losing the baseline | Tests red for 20 minutes | Verify every checkpoint |
| Silent driving | No narration | Narration replay |
| Assuming the policy | Using AI where it is banned | Ask the recruiter |
Self-Assessment Rubric
| Level | Standard |
|---|---|
| L0 | No plan; delegates wholesale; accepts unread diffs; tests red at the end |
| L1 | Has a plan; delegates in chunks; reads diffs; green at the end but not throughout |
| L2 | Checkpoint plan; green throughout; rejects bad output with specific reasons; narrates |
| L3 | Above, plus takes over at the right boundary and says why; rejects a plausible-looking diff for a subtle reason; finishes with a smaller verified diff and a stated plan for the remainder |
Hire-bar translation
| Verdict | What it looks like |
|---|---|
| No hire | Prompt-and-hope; unverified diff at time |
| Hire (senior) | Planned, chunked, verified, green |
| Strong hire (senior) | Above, plus a specific rejection and clear narration |
| Hire (staff) | Above, plus correct manual-takeover judgement, stated aloud |
| Strong hire (staff) | Above, plus improves the repo's verification story — adds the test that makes the next change safe |
References
../../research/findings.md— the corroboration../../research/source-report.md— rows 37–40../coding/README.md— the progressive-gate skill this shares- Exponent. Google's AI-Assisted Coding Interview (2026 Guide). https://www.tryexponent.com/blog/google-ai-coding-interview
- interviewing.io. How to use AI in Meta's AI-assisted coding interview. https://interviewing.io/blog/how-to-use-ai-in-meta-s-ai-assisted-coding-interview-with-real-prompts-and-examples
- Related track in this repo: agentic-engineer — agent infrastructure from the builder's side, which is useful background for reasoning about what the agent is actually doing