Track G — Warmup: Driving an Agent, Worked

Self-contained. The method, a fully worked 60-minute transcript with commentary, the specific diffs to reject and why, and the scoring rubric applied to a real run.

Reported as a beta fifth round; independently corroborated as an industry-wide format. Do not skip it.

Table of Contents

• Chapter 0: What Is Actually Being Scored
• Chapter 1: The Method, In Eight Steps
• Chapter 2: A Worked 60 Minutes
• Chapter 3: Five Diffs, And Whether To Accept Them
• Chapter 4: The Takeover Boundary
• Chapter 5: Prompting That Works Here
• Chapter 6: Scoring That Run
• Chapter 7: Ask The Recruiter First
• References

Chapter 0: What Is Actually Being Scored

Not "can you use an AI". Everyone can. The scored skill is engineering judgement applied to a fast, confident, occasionally wrong collaborator — which is a real and specific skill with observable behaviours.

The tell that separates the top quartile: the candidate who rejects an agent's output and says exactly why. Someone who accepts every diff is not reviewing, and from the interviewer's side they are indistinguishable from someone who does not understand the code. One specific rejection is worth more than three accepted diffs.

Note also that the reported format — a multi-file codebase with phased objectives: bug fix, then implementation, then optimization — is the same progressive-gate structure as Track A. The gating skill transfers directly. What is new is driving a second agent through it while narrating.

Chapter 1: The Method, In Eight Steps

1. Establish the baseline before touching anything

python3 -m pytest -q 2>&1 | tail -3

Record the test count and the runtime. You cannot claim you kept it green if you never knew it was green, and "the suite was already failing" is a thing you want to discover at minute one rather than minute forty.

Narrate it: "First thing, baseline — 412 tests, 8 seconds, all passing. That's my invariant."

2. Read enough to write a plan — ten minutes maximum

Enough to name the files that will change and the invariant that must hold. Not enough to understand everything; you will not have time, and the round knows that. Reading unfamiliar code under pressure is half of what is being tested.

What to look for, in order: the entry point for the feature, the test file that covers it, and the seam where your change goes.

3. Write the plan before invoking the agent

In a scratch file or out loud. It must contain:

• The end state, in one sentence.
• Three to five checkpoints, each independently verifiable.
• The invariant — usually "this test file keeps passing".
• What you will do manually rather than delegate.

Then narrate the delegation decision, because it is a large share of the score:

"I'm going to do the mechanical rename with the agent — it's 30 files and the test suite verifies it completely. But I'm writing the state-machine change myself, because the invariant is subtle and I want to be the person who understands it."

4. Delegate in checkpoint-sized units

Not "implement the feature". Not "fix line 42". One verifiable unit:

"Add a timeout parameter to these four public functions in client.py, thread it through to transport.py, and keep tests/test_timeout.py passing. Do not change any default behavior — a call with no timeout must behave exactly as today."

That prompt contains the scope, the boundary, and the invariant. Compare with "add timeouts", which contains none of them.

5. Verify every checkpoint before proceeding

Run the tests. Read the diff. Read the diff even when the tests pass — a passing suite proves you did not break what was tested, not that you built what you meant.

6. Reject bad work, specifically

"This adds a bare except Exception in the retry path. That will swallow CancelledError, which is a BaseException — so the client becomes uncancellable and our shutdown path hangs. Catch httpx.TransportError instead."

Specific, technical, names the consequence. That is a code review, and it is what the round is for.

7. Take over when the agent is looping

Two failed attempts at the same checkpoint means the problem is under-specified or genuinely subtle. Write it yourself and say why:

"It's fighting me here because the constraint isn't expressible in the test — I'll do this one directly."

Recognizing that boundary is a senior signal; grinding through a third attempt is not.

8. Narrate continuously

Same rule as coding rounds. Silence reads as lost. Say what you are delegating and why, what you are checking, and what you rejected.

Chapter 2: A Worked 60 Minutes

Setup: httpx-like HTTP client library, ~25k LOC, 412 tests. Task: add per-request retry with configurable backoff, wire it through the sync and async client paths, and expose it in the public API. Too large to hand-write in an hour.

Commentary in the right column is what the interviewer is scoring.

00:00 — Baseline

$ python3 -m pytest -q
412 passed in 7.9s

"412 tests, 7.9 seconds, green. That's my invariant for the whole hour — if I break it, I stop and fix it before continuing."

Scored: baseline established before touching anything. Invariant named out loud.

00:02 — Reading, with a timer

"I'm giving myself eight minutes to read. I want three things: where a request actually gets sent, where the sync and async paths diverge, and what the existing timeout handling looks like — because retries are going to interact with it."

$ ls src/client/
_client.py  _transports/  _config.py  _exceptions.py  _models.py
$ grep -rn "class Timeout" src/
$ grep -rn "def send" src/client/_client.py

Scored: time-boxed reading. Specific questions rather than browsing. The observation that retries interact with timeouts is the kind of thing that shows domain judgement.

00:10 — The plan, written down and narrated

*"Here's my plan. End state: a Retry config object on the client, honoured by both sync and async send paths, with exponential backoff plus jitter, retrying only on transport errors and a configurable set of status codes.

Four checkpoints:

1. Retry config class plus validation, with unit tests. Nothing wired.
2. Wire it into the sync path only. Existing tests must stay green.
3. Wire it into the async path.
4. Public API surface and docs.

Invariant throughout: all 412 existing tests pass, and no existing default behaviour changes — a client constructed with no retry config must behave byte-identically to today.

Delegation split: I'll have the agent do checkpoints 1 and 4 — config classes and API surface are mechanical and the tests verify them completely. I'll write the backoff-with-jitter logic myself, because getting jitter wrong is the difference between smoothing retries and building a synchronized retry storm, and I want to own that. Checkpoints 2 and 3 I'll do with the agent but review closely, because that's where the cancellation semantics live."*

Scored heavily. This is the single highest-value 90 seconds of the round: end state, verifiable checkpoints, explicit invariant, and a reasoned delegation split with a stated reason for the manual part.

00:12 — Checkpoint 1, delegated

Prompt: "In src/client/_config.py, add a Retry dataclass with fields: max_attempts: int = 3, backoff_base: float = 0.2, backoff_cap: float = 30.0, retry_on_status: frozenset[int] = frozenset({429, 502, 503, 504}). Validate in __post_init__: `max_attempts

= 1, backoff_base > 0, backoff_cap >= backoff_base, raising ValueErrorwith a message naming the field. Addtests/test_retry_config.py` covering each validation failure and the defaults. Do not modify any other file."

Scored: the prompt states scope ("do not modify any other file"), the exact contract, and the test expectation. It is verifiable in one command.

00:15 —

$ python3 -m pytest -q
420 passed in 8.1s

"420 now, so eight new tests, and the original 412 are intact. Let me read the diff before I move on — green doesn't mean right."

Reads the diff.

"Two things. First, it used set rather than frozenset for the default, which is a mutable default shared across instances — classic bug, and dataclass would normally catch it but it didn't here because the default came from a factory. Second, the error messages are generic. I'll fix the first myself since it's one word; the second is fine for now."

Scored: read the diff despite green tests, found a real defect the tests did not catch, and made a proportionate decision about each.

00:18 — Checkpoint 2, the manual part

"I'm writing the backoff function myself."

def _sleep_for(self, attempt: int) -> float:
    """Full jitter: uniform over [0, min(cap, base * 2**attempt)].

    Not equal jitter and not fixed backoff — AWS's published simulation found
    full jitter minimized both total work and completion time under contention.
    Fixed backoff synchronizes every client that failed at the same instant,
    which turns a partial outage into a retry storm.
    """
    ceiling = min(self.backoff_cap, self.backoff_base * (2 ** attempt))
    return self._rng.uniform(0, ceiling)

"I'm injecting the RNG so this is deterministically testable. A retry policy you can't test deterministically is one you can't ship."

Scored: took the subtle part personally, as planned. The docstring explains why this variant. The injected RNG shows testability thinking.

00:24 — Checkpoint 2, delegated wiring

Prompt: "In src/client/_client.py, wrap Client.send so that when self._retry is not None, transport errors and responses with status in retry_on_status are retried up to max_attempts, sleeping self._sleep_for(attempt) between attempts. Requirements: (a) if self._retry is None the code path must be byte-identical to today; (b) do not retry on 4xx other than those listed; (c) the response body of a retried response must be closed before retrying, or we leak connections; (d) all 420 existing tests must still pass. Add tests/test_retry_sync.py."

Scored: requirement (c) is domain knowledge the agent may well miss, stated up front. Requirement (a) protects the invariant explicitly.

00:29 —

$ python3 -m pytest -q
3 failed, 429 passed

"Three failures. Let me look before re-prompting — 'that didn't work' is not a useful instruction to anyone."

FAILED tests/test_timeout.py::test_timeout_not_retried
FAILED tests/test_client.py::test_send_no_retry_config_unchanged
FAILED tests/test_retry_sync.py::test_retries_on_503

"This is informative. test_timeout_not_retried says a timeout is being retried, and it shouldn't be by default — a timeout usually means the server got the request, so retrying a non-idempotent call could double a side effect. And test_send_no_retry_config_unchanged means it broke requirement (a). So the agent treated TimeoutException as a transport error, and it restructured the no-retry path instead of leaving it alone. Both are my prompt's fault for being imprecise, not the agent's."

Scored heavily: diagnosed the failures rather than re-prompting blindly, understood why each is wrong at a domain level, and attributed the failure to their own under-specification. That last part is genuinely rare and reads very well.

00:32 — Re-prompt, precisely

Prompt: "Two corrections. First, TimeoutException must not be retried by default — a timeout means the request may have been received, so retrying a non-idempotent request could duplicate a side effect. Add retry_on_timeout: bool = False to Retry and gate it. Second, requirement (a) is violated: test_send_no_retry_config_unchanged fails. Restore the original code path exactly when self._retry is None — put the retry logic in a separate method and branch to it, rather than restructuring send itself."

00:36 —

$ python3 -m pytest -q
433 passed in 8.4s

"Green, and the count went up by 4 — the new timeout-gating tests. Reading the diff."

"This is better. The branch is clean. One thing I'm going to change: it's calling response.close() inside the retry loop but not in a finally, so if _sleep_for raises — which it won't today, but it's a public-ish method — we leak the connection. I'll move it."

Scored: found a latent bug the tests do not cover, and stated the reasoning about why it matters even though it cannot happen today.

00:40 — Checkpoint 3, the async path

"This is the one I'm most worried about, so I'm going to be specific about cancellation up front rather than fixing it afterwards."

Prompt: "Mirror the retry logic in AsyncClient.send in the same file. Three requirements beyond the sync version: (a) use await anyio.sleep(...), not time.sleep; (b) do not catch asyncio.CancelledError or any BaseException — only the specific transport exception types, because CancelledError is a BaseException and swallowing it would make the client uncancellable; (c) await response.aclose() before retrying. Add tests/test_retry_async.py including a test that cancels a request mid-retry and asserts the CancelledError propagates."

00:47 —

$ python3 -m pytest -q
441 passed in 9.1s

"Green. Reading the async diff especially carefully, because this is where the subtle bugs live."

"Good — it used except (TransportError, ConnectError) specifically, and the cancellation test is real: it cancels during the sleep and asserts propagation. That's the test I actually wanted."

Scored: the cancellation constraint was specified before the code was written, which is much better than catching it in review. And the review verifies the test is meaningful, not just present.

00:50 — Checkpoint 4, and a rejection

Prompt: "Expose retry in the public API: add the retry: Retry | None = None parameter to Client.__init__ and AsyncClient.__init__, export Retry from src/client/__init__.py, and add a section to docs/advanced.md with a working example."

00:54 —

"Tests are green but I'm rejecting part of this diff."

*"It added retry=Retry() as the default rather than None. That silently turns retries on for every existing user of this library on upgrade. That's a breaking behavioural change disguised as a default — someone's non-idempotent POST starts getting retried after a patch bump. Default must be None.

Second, the docs example uses max_attempts=10 with backoff_base=1.0, which is a worst case of about 17 minutes of retrying. That's not a sensible example to put in front of users; people copy examples. Change it to the defaults."

Scored very heavily. Both rejections are about consequences, not style; the tests passed for both; and the first is exactly the class of bug — a plausible-looking default that changes behaviour for existing users — that a reviewer who is not reading carefully would wave through.

00:58 — Close out

$ python3 -m pytest -q
441 passed in 9.0s

*"Where I am: 441 tests, up from 412, all green, and green at every checkpoint along the way. Retry config, sync path, async path with correct cancellation semantics, and the public API.

What I did not do: I haven't added retry to the streaming path, because streaming retries need to handle partially-consumed bodies and that's a genuinely different problem — I'd want to design it rather than bolt it on. I also haven't added a retry budget, which I'd want before shipping this: exponential backoff with jitter bounds the rate of retries from one client, but it doesn't bound the total amplification across a fleet. Three attempts at a 95% failure rate is still 2.85× offered load onto a dependency that's already failing. That's the next thing I'd build.

Riskiest line in what I did write: the response-close handling in the sync retry loop. It's correct now, but it's the place where a future change is most likely to introduce a connection leak, and it isn't directly tested."*

Scored: stated what was not done and why · identified a genuine architectural gap (retry budget) that the task did not ask for · volunteered the riskiest line in their own work unprompted, which is the single most credibility-generating move available.

Chapter 3: Five Diffs, And Whether To Accept Them

Practice material. For each: accept, reject, or accept-with-change — and say why.

Diff 1

  try:
      response = self._transport.handle_request(request)
- except httpx.TransportError:
+ except Exception:
      if attempt < self.max_attempts:
          continue
      raise

REJECT. except Exception catches programming errors — TypeError from a bug in request construction — and retries them three times before surfacing, turning an instant clear failure into a slow confusing one. In the async version it is worse: it would catch anything derived from Exception and hide real bugs. Catch the specific transport types. (Note it does not catch CancelledError, which is a BaseException — but that is luck, not intent.)

Diff 2

+ def test_retry_on_503(self):
+     client = Client(retry=Retry(max_attempts=3))
+     with mock.patch.object(client._transport, "handle_request") as m:
+         m.return_value = Response(503)
+         client.get("https://example.com")
+     assert m.call_count == 3

ACCEPT WITH CHANGE. The assertion is real — it pins the retry count, which is the behaviour under test. But it does not assert what the client finally returns or raises after exhausting attempts, which is the part a user actually experiences. Add with pytest.raises(...) or assert the returned response is the last 503. As written it would pass even if the client swallowed the failure and returned None.

Diff 3

+ import time
+
  async def _retry_send(self, request):
      for attempt in range(self.max_attempts):
          try:
              return await self._transport.handle_async_request(request)
          except TransportError:
+             time.sleep(self._sleep_for(attempt))
              continue

REJECT, and this is the important one. time.sleep in an async function blocks the entire event loop. Every other request in the process stalls for the backoff duration — and backoff grows exponentially, so a client retrying at attempt 5 with a 30-second cap stalls everything for up to 30 seconds. Must be await anyio.sleep(...).

This diff would pass every test in a suite that does not measure concurrency, which is why the review matters more than the suite.

Diff 4

- def __init__(self, *, timeout=DEFAULT_TIMEOUT, retry=None):
+ def __init__(self, *, timeout=DEFAULT_TIMEOUT, retry=Retry()):

REJECT. A behavioural change disguised as a default. Every existing user gets retries turned on when they upgrade — including on non-idempotent requests, where a retry can duplicate a side effect. Opt-in behaviour must default to off. This is the diff most likely to be waved through, because it looks like a convenience.

(Bonus defect: Retry() as a default argument is a single shared instance evaluated once at definition time. If Retry ever becomes mutable, every client shares state.)

Diff 5

+ for attempt in range(self.max_attempts):
+     try:
+         return self._send_once(request)
+     except TransportError:
+         if attempt == self.max_attempts - 1:
+             raise
+         time.sleep(self.backoff_base * (2 ** attempt))

ACCEPT WITH CHANGE. The structure is right — it re-raises on the final attempt rather than falling out of the loop and returning None, which is a real bug it avoided. But the backoff has no jitter and no cap. No jitter means every client that failed at the same instant retries at the same instant, synchronizing the herd. No cap means attempt 10 sleeps for 102 seconds. Use min(cap, base * 2**attempt) and wrap it in random.uniform(0, ...).

Chapter 4: The Takeover Boundary

Knowing when to stop delegating is a scored judgement. The rules:

Delegate when:

• The work is wide and mechanical — a 30-file rename, a signature change across call sites.
• The test suite verifies it completely.
• You could do it yourself but it would take 20 minutes of typing.
• It is boilerplate whose shape you can specify precisely.

Take over when:

• Two attempts have failed on the same checkpoint. The problem is under-specified or subtle; a third attempt is a bet against evidence.
• The invariant is not expressible in a test. If you cannot write the assertion, you cannot give the agent a checkpoint, and you are hoping rather than verifying.
• It is the part you need to understand to answer questions about later.
• It is security- or correctness-critical in a way where a plausible-looking wrong answer is worse than slow progress.

The narration that earns the point:

"It's fighting me on this because the constraint isn't expressible in the test — the property I need is 'no two concurrent calls observe the same token', and I can't write a non-flaky assertion for that quickly. I'll write this one directly."

Do not take over the 30-file rename. That is the inverse error and it is just as visible: you are spending the round's scarcest resource — time — on the work most suited to delegation.

Chapter 5: Prompting That Works Here

Not tricks. The same properties that make a good ticket.

The last two rows are where your value is. The agent can write a retry loop. It does not know that your transport leaks connections if the body is not closed, or that your shutdown path depends on cancellation propagating. Front-loading that knowledge is the difference between one attempt and three.

When re-prompting after a failure, never say "that didn't work". Say what failed, why it is wrong at a domain level, and what to do instead:

❌ "The tests are failing, please fix." ✅ "test_timeout_not_retried fails because you treated TimeoutException as retryable. A timeout means the server may have received the request, so retrying a non-idempotent call can duplicate a side effect. Add retry_on_timeout: bool = False and gate on it."

Chapter 6: Scoring That Run

Against the Track G rubric:

Level: L3. Hire-bar: strong hire (staff).

The two things that put it there rather than at hire (senior):

1. The Retry()-default rejection. Tests passed. It looked like a convenience. Catching that it silently changes behaviour for every existing user on upgrade is the review judgement the round exists to measure.
2. Volunteering the riskiest line and the missing retry budget at the end. Identifying an architectural gap the task did not ask about, and naming the weakest part of your own work before anyone finds it, is the strongest available signal — and it costs thirty seconds.

Chapter 7: Ask The Recruiter First

Policies at companies you may interview at in the same month are opposite.

Ask, per company, per round. It is not a question that makes you look unprepared — it makes you look like someone who reads the rules. It goes on the pre-interview checklist in ../../STATE.md.

References

• README.md — Track G tasks, drills, failure modes, rubric
• ../../research/findings.md — the corroboration that this format is industry-wide
• ../coding/README.md — the progressive-gate skill this shares
• ../python-internals/WARMUP.md#33-cancellation — why the CancelledError constraint in the transcript matters
• Exponent. Google's AI-Assisted Coding Interview (2026 Guide). https://www.tryexponent.com/blog/google-ai-coding-interview
• interviewing.io. How to use AI in Meta's AI-assisted coding interview. https://interviewing.io/blog/how-to-use-ai-in-meta-s-ai-assisted-coding-interview-with-real-prompts-and-examples
• Brooker, M. Exponential Backoff and Jitter. https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/ — the full-jitter argument used in the transcript
• Related track: agentic-engineer — agent infrastructure from the builder's side