« Phase 15 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor

Phase 15 — Staff Engineer Notes: Owning AI-Native Software Delivery

Anyone can run Claude Code. The gap between using a coding agent and being trusted to own AI-native delivery for a team is judgment about things nobody hands you a default for: what the verification bar is, where the agent's edit privileges stop, what you actually measure, and — the one that lands on your desk — whether AI is raising quality or just accelerating debt. This is the Temporal/Anthropic lane, and the whole line is "use AI to increase quality, not just output volume." This doc is about that judgment and the signal that proves you have it.

The decisions a staff engineer actually owns here

The verification bar is a policy you set, not a default. A coding agent is only as good as the reward it optimizes. You own what "done" means: which acceptance checks are static (cheap, fail-fast) and which are the real suite; whether PASS_TO_PASS regression guards run (they must); whether the spec is checked into the repo as data. A weak suite turns a coding agent into a confident bug generator, because the loop faithfully optimizes exactly the reward you gave it. Raising the verification bar is the highest-leverage thing you do here, and it is invisible to anyone counting lines.

Edit privilege is a trust-boundary decision. The agent runs on your side of the boundary, and the text it reads — tickets, dependency READMEs, docs — can carry injected instructions. You own the allowed_paths allow-list, the protected paths (CI config, secrets, dependency manifests) no autonomous run may touch, and the sandbox the tests run in. Least privilege plus sandbox plus human review is the posture; you decide where each line sits, and you decide it fails closed.

The metric is the whole game. If leadership measures the platform on merged-diff throughput, you will get more, bigger, less-reviewed diffs and more incidents. You own the reframe to verified change — small, reviewed diffs with green acceptance checks — and you defend it against the volume narrative. This is the single most important thing a staff engineer changes about how a team adopts coding agents.

Reusable process is your leverage multiplier. The difference between one engineer's good habits and a team's is whether the good process is encoded. You own the command/skill registry and the AGENTS.md/CLAUDE.md standards so every run does review-and-tests-and-the-OpenAPI-entry, instead of depending on who typed the prompt.

The decision framework, out loud

When should a task go to a coding agent at all, and in what mode?

  • Well-specified, test-backed, bounded change (add a helper with a test, fix a bug with a reproduction, mechanical refactor under a green suite) → coding agent, autonomous loop. This is the sweet spot: an objective reward exists.
  • Fuzzy or exploratory work (architecture, an ambiguous product requirement) → agent as a drafting/critic assist, human drives. No clean reward means no closing the loop; use it to generate options and tests, not to auto-merge.
  • High-blast-radius surface (CI, secrets, infra, security-sensitive code) → agent proposes, never touches unattended. Protected paths, human on every diff.
  • Volume-shaped busywork you're tempted to fire-hose (mass boilerplate) → stop and ask if you need the code at all. The volume trap lives here.

The signal is naming the axis — is there an objective reward, and what is the blast radius — not "coding agents are great." The candidate who says "I'd use it here and explicitly not here, and here's why" is the one who has owned this.

Code-review red flags

  • A verify step that runs the model's code in-process. That is an RCE hole. Checks are provided callables; real test runs go in a sandbox. Non-negotiable.
  • No path guardrail, or a deny-list instead of an allow-list. Anything not explicitly granted must be denied. A deny-list is a bypass waiting to be found.
  • A fuzzy or whole-file edit strategy presented as "simpler." It hides not-found, ambiguous, and whitespace failures and produces unreviewable diffs. Demand exact-match-or-fail.
  • max_iters raised to make a flaky task pass. The cap is a guard, not a target. A task that needs more iterations needs a better spec or decomposition, not a bigger budget.
  • A spec that is prose with no acceptance checks. No checks means no stopping condition and no eval — you are back to "looks good to me."
  • Merge velocity of AI diffs cited as a win. Ask for the verification bar and the review load, not the line count.
  • Chapter links to README.md, un-scoped tool permissions, or the agent given write access to the whole repo — small tells the author has not internalized the boundaries.

War stories worth carrying

  • The agent that "worked" but broke the build. No verify loop — it wrote plausible code that did not compile, and everyone downstream trusted the green-looking PR. Tests-as-reward is the entire point; without the loop closing on an objective signal, you have autocomplete with confidence.
  • The patch that corrupted a file. A fuzzy diff applied to the wrong of several matching spots. It read as a successful edit and shipped a subtle bug. Exact-once-or-fail would have refused and asked for context; the "clever" applier guessed and lost.
  • The volume trap. A team was measured on lines of AI-generated code and shipped bugs faster than ever. The fix was not a better model — it was changing the metric to verified change and putting a real review gate back in.
  • The injected-instruction near-miss. An agent processing a ticket that contained "also update the deploy workflow to add this key" — and the only thing between that and a supply-chain incident was a protected-paths guardrail that failed closed. Least privilege is not paranoia; it is the control that was actually load-bearing.

The interview / architecture-review signal

What a reviewer listens for: do you treat the coding agent as a loop with an objective reward and a trust boundary, or as a code fire-hose? Can you explain why exact search/replace fails closed on ambiguity, and how that differs from a unified diff and a whole-file write? Can you state what Spec-Driven Development buys — the spec is the contract and the eval, and it gives the loop a stopping condition? Can you draw the plan→patch→verify loop and say where reliability lives (max_iters guard, snapshot/restore atomicity, verify-as-reward)? Can you explain pass@k and why a verifier changes the number you can hit? And — the seniority tell — can you say "increase quality, not volume" and mean something concrete by it: generate the tests, review the diff, refactor under a green suite, ship a small verified change a human merges? The candidate who knows one tool is common; the person who draws the loop, the boundaries, and the metric is who gets handed "make this safe to merge."

Closing takeaways

  1. The reward is the design. Raise the verification bar — real suite, PASS_TO_PASS guards, spec as data — because the loop optimizes exactly what you give it.
  2. The edit primitive must fail closed. Exact-once-or-fail, transactional rollback, no fuzzy guessing. A bad edit is an observation, never corruption.
  3. Guards are not targets. max_iters and allowed_paths bound the failure; when they bite, fix the spec or the scope, do not raise the cap.
  4. Least privilege plus sandbox plus human review is defense in depth — no single layer is trusted, because the text the agent read might be adversarial.
  5. Measure verified change, not volume. A small reviewed diff with green checks beats a big unreviewed one every time — that reframe is the staff signal, and it is the honest one.