Track G — Agentic Coding

Reported as a beta fifth round, not administered to everyone: an existing codebase plus a problem too large to hand-write in the time, worked through using an AI coding agent (../../research/source-report.md rows 37–40).

The research says do not skip this. The source report frames it as an optional beta, which invites skipping. Independent corroboration says the format is going industry-wide.

→ Study guide: WARMUP.md — the method in eight steps, a fully worked 60-minute transcript with scoring commentary, and five diffs to accept or reject.

DIFFBANK.md — 30 agent diffs, 90 seconds each. The round is scored on which of the agent's output you let through, and that is a trainable skill: the six-pass review, the ranked taxonomy of what agents actually get wrong, and three diffs that look wrong and are right (because rejecting everything is also a failure).


Table of Contents


Why This Is Not Optional

Corroborated independently of the source report (../../research/findings.md): Meta began rolling out AI-enabled coding interviews in late 2025; Google is piloting a Gemini-assisted coding format; CodeSignal shipped agentic coding assessments as a product.

The reported common shape: 60 minutes, a multi-file codebase, phased objectives — bug fix, then core implementation, then optimization — with the assistant confined to a chat panel rather than given direct file-edit authority. Stated evaluation criteria converge on AI fluency: prompt construction, output validation, and debugging the assistant's work.

Note that the phased structure is the same progressive-gate format as Track A. The gating skill transfers directly; what is new is driving a second agent through it while narrating your strategy.


Ask the Recruiter First

Policies at companies you may interview at in the same month are opposite.

CompanyReported policy
OpenAIAgentic round in beta, some candidates
Meta, GoogleActively piloting AI-assisted rounds
AnthropicReportedly prohibits AI tools in live interviews; candidates reportedly removed for using them

So: ask, per company, per round. Never assume, in either direction. This goes on the pre-interview checklist in ../../STATE.md, and it is not a question that makes you look unprepared — it makes you look like someone who reads the rules.


What Is Actually Being Scored

Not "can you use an AI." Everyone can. The scored skill is engineering judgement applied to a fast, confident, occasionally wrong collaborator — which is a real and specific skill.

ScoredNot scored
Decomposing an oversized problem into verifiable unitsTyping speed
Writing a plan before invoking the agentPrompt "tricks"
Giving the agent checkpoints you can actually verifyVolume of code produced
Reading diffs critically and rejecting bad onesAccepting everything that compiles
Keeping tests green throughoutGreen at the end only
Knowing when to take over manuallyDelegating everything on principle
Narrating the strategy out loudSilent driving

The tell that separates the top quartile: the candidate who rejects an agent's output and says why. Someone who accepts every diff is not reviewing, and the interviewer cannot distinguish them from someone who does not understand the code. Rejecting one plausible-looking diff with a specific reason is worth more than three accepted ones.


The Environment

cd tracks/agentic
# Clone a mid-size, well-tested pure-Python repo with a real test suite.
# Good properties: 10k-60k LOC, fast tests, no heavy native deps, active history.
git clone --depth 50 <target-repo> workspace
cd workspace && python3 -m pytest -x -q       # establish the green baseline FIRST

Suggested targets — pick one you have not worked in, because reading unfamiliar code under time pressure is half the round:

RepoWhy it works
httpxAsync + sync dual API, clean layering, fast suite
pydantic (v1 branch)Type machinery, lots of edge cases
richRendering pipeline, plenty of pure-logic surface
sqlglotParser/transpiler — dense, well-tested, great for refactors
flask / clickSmall, canonical, good for cross-cutting changes

Record the baseline: test count, runtime, and coverage if available. Every task below is scored partly on whether that baseline stayed green the entire time, not just at the end.


The Six Tasks

Each is deliberately too large to hand-write in 60 minutes. That is the point — the constraint is what forces delegation, and delegation is what is being scored.

#TaskShapeSkill it isolates
G1Cross-cutting refactor — change a signature or pattern used in 30+ call sites, preserving behaviorWide, shallowBatching mechanical work; verifying breadth you cannot read
G2Feature spanning modules — add a capability touching parsing, core logic, and public APIDeep, narrowDecomposition; sequencing; keeping the suite green mid-flight
G3Performance fix requiring profiling — find the hot path, fix it, prove the improvementInvestigativeMaking the agent measure rather than guess
G4Test-coverage expansion — take an under-tested module from 40% to 85% with tests that would actually catch bugsGenerativeRejecting assertion-free tests; the agent's default failure mode
G5Bug in unfamiliar code — a real reverted commit, re-applied. Find and fix it from a failing testDiagnosticDirecting an investigation, not a code generation
G6Migration — move a subsystem to a new API or library, with a deprecation pathBroad + riskyPlanning; incremental verification; knowing when to take over

Each gets a directory holding: your written plan, the transcript, the final diff, the test log, and your self-score.


The Method

This is the thing to internalize. It is the difference between driving and hoping.

1. Establish the baseline before touching anything

Run the tests. Record the count and the runtime. You cannot claim you kept it green if you never knew it was green.

2. Read enough to write a plan

Ten minutes, maximum, of reading. Enough to name the files that will change and the invariant that must hold. Not enough to understand everything — you will not have time, and the round knows that.

3. Write the plan before invoking the agent

In the chat, out loud, or in a scratch file. It must contain:

  • The end state, in one sentence
  • The checkpoints — three to five, each independently verifiable
  • The invariant that must hold throughout (usually: this test file keeps passing)
  • What you will do manually rather than delegate

Narrate this. "I'm going to do the mechanical rename with the agent because it's 30 files and I can verify it with the test suite, but I'm writing the state-machine change myself because the invariant is subtle and I want to be the one who understands it." That sentence alone is a large share of the score.

4. Delegate in checkpoint-sized units

Not "implement the feature." Not "fix line 42." One verifiable unit: "Add the timeout parameter to these four public functions, thread it through to the transport layer, and keep tests/test_timeout.py passing. Do not change the default behavior."

5. Verify every checkpoint before proceeding

Run the tests. Read the diff. Read the diff even when the tests pass — a passing suite proves you did not break what was tested, not that you built what you meant.

6. Reject bad work, specifically

"This adds a bare except Exception in the retry path, which will swallow CancelledError and make the client uncancellable. Use except httpx.TransportError instead." Specific, technical, cites the consequence. That is a review, and it is what the round is for.

7. Take over when the agent is looping

Two failed attempts at the same checkpoint means the problem is under-specified or genuinely subtle. Write it yourself and say why: "It's fighting me on this because the constraint isn't expressible in the test — I'll do it directly." Recognizing that boundary is a senior signal; grinding through a third attempt is not.

8. Narrate continuously

Same rule as Track A. Silence reads as lost. Say what you are delegating and why, what you are checking, and what you rejected.


The Good-vs-Bad Rubric

DimensionSomeone who pastes prompts and hopesSomeone driving
Before startingTypes the task into the agentRuns tests, reads, writes a plan with checkpoints
Unit of delegationThe whole problemOne verifiable checkpoint
Prompting"Make it work"States the invariant, the boundary, and what not to change
On outputAccepts if it compilesReads the diff; runs tests; questions anything unexplained
On failureRe-prompts with "that didn't work"Diagnoses why, then re-scopes or takes over
TestsGreen at the end, maybeGreen at every checkpoint
Test qualityAccepts whatever the agent wroteRejects tests that assert nothing meaningful
Manual workNever, on principleTakes the subtle parts personally
NarrationSilentContinuous strategy commentary
At timeA large diff, unverifiedA smaller diff, verified, with a stated plan for the rest

A smaller verified diff beats a larger unverified one. That is the whole rubric in one line, and it maps directly onto Track A's "get something correct early" — an unverified change is an unopened gate.


Drill Set

DrillCadenceTrains
Timed task runWeekly, 60 minThe round. One task, recorded, scored
Plan-only, 10 min3×/weekRead a task, write the checkpoint plan. No agent
Diff review2×/weekTake an agent diff and find three things wrong with it
Rejection practiceWeeklyDeliberately accept a bad diff, run tests, find what broke. Calibrates trust
Takeover judgementWeeklyNote every point you considered taking over. Review whether you were right
Narration replayWeeklyListen back. Was the strategy audible?
Cold repoMonthlyA repo you have never opened. Reading speed under pressure is the hidden variable

Failure Modes

FailureSymptomFix
No planStraight to promptingPlan-only drill
Delegating the whole problemOne giant prompt, one giant diffCheckpoint-sized units
Not reading diffsAccepts anything greenDiff-review drill
Accepting empty testsCoverage up, quality flatG4 specifically; assert on behavior, not on calls
Never taking overThird attempt at the same checkpointTwo-strike rule
Taking over too earlyHand-writing the mechanical 30-file renameThat is exactly what to delegate
Losing the baselineTests red for 20 minutesVerify every checkpoint
Silent drivingNo narrationNarration replay
Assuming the policyUsing AI where it is bannedAsk the recruiter

Self-Assessment Rubric

LevelStandard
L0No plan; delegates wholesale; accepts unread diffs; tests red at the end
L1Has a plan; delegates in chunks; reads diffs; green at the end but not throughout
L2Checkpoint plan; green throughout; rejects bad output with specific reasons; narrates
L3Above, plus takes over at the right boundary and says why; rejects a plausible-looking diff for a subtle reason; finishes with a smaller verified diff and a stated plan for the remainder

Hire-bar translation

VerdictWhat it looks like
No hirePrompt-and-hope; unverified diff at time
Hire (senior)Planned, chunked, verified, green
Strong hire (senior)Above, plus a specific rejection and clear narration
Hire (staff)Above, plus correct manual-takeover judgement, stated aloud
Strong hire (staff)Above, plus improves the repo's verification story — adds the test that makes the next change safe

References