« Phase 15 README · Warmup · Hitchhiker's · Deep Dive · Core Contributor · Staff Notes

Phase 15 — Principal Deep Dive: The Coding Agent as a Production SDLC Platform

The Deep Dive traced the loop. This doc is about the system that loop lives inside: how a coding agent becomes a fleet in your CI, where it scales and where it stalls, which failures have a blast radius that reaches production, and which of its "looks wrong" decisions are load-bearing. The through-line, straight from the Temporal JD: the platform's value is verified change, not code volume — a coding-agent platform that optimizes throughput of merged diffs is an incident generator.

The loop is one worker; the platform is everything around it

A single plan → patch → verify run is a stateless function: (spec, repo-snapshot, model) → (verified-diff | did-not-converge, transcript). A production coding-agent platform is that function plus five surrounding systems, and the interesting architecture is in the seams:

• A spec store. Spec-Driven Development means the contract-and-eval (spec.md / plan.md / tasks.md, AGENTS.md, CLAUDE.md) lives in the repo, versioned with the code. The spec is data, not a prompt typed into a chat, so it is reviewable, diffable, and reusable.
• A command/skill registry. Parameterized workflows (name + args → Spec) checked into .claude/commands/, so every run of "add an endpoint with a test and an OpenAPI entry" executes the same good process instead of whatever the operator remembered to type.
• A sandboxed verifier fleet. The acceptance checks are the real test suite, run in ephemeral, network-isolated Phase 09 sandboxes — never in the agent's process, never on a developer laptop with ambient credentials.
• A review gate. The transcript is the diff a human approves. The model proposes; a person merges. This is the non-negotiable control, not a nicety.
• An orchestrator. Schedules runs against a repo at a commit, enforces max_iters and wall- clock budgets, and fans out parallel samples for pass@k.

The architectural discipline: the loop is pure and deterministic given the model, so the platform is testable by stubbing the model — exactly as the lab injects planner/patcher. A platform whose control-flow correctness depends on the live model is a platform you cannot regression-test.

The capacity envelope: the verifier, not the model, sets it

There is no single "coding-agent throughput" number; the envelope is a composition, and the counter-intuitive fact is that the test suite, not the LLM, usually dominates cost and latency.

• Per-run cost is iterations × (model tokens + verify runtime). Model latency is seconds; a real repo's test suite can be minutes. Above a couple of iterations the verify runtime dominates wall-clock, which is why "just raise max_iters" is the wrong lever — you multiply the expensive part.
• pass@k is a capacity-for-correctness trade. Run the agent k times and accept the task if any run's tests pass. The unbiased estimator (given n ≥ k samples, c passing) is pass@k = 1 − C(n−c, k) / C(n, k). The verifier is what makes this usable: it lets you spend more samples and keep only the correct one. pass@1 is what a user feels; pass@10 is what an agent with a verifier can reach. That gap is the entire economic case for closing the loop on an objective reward — and it is a spend decision: sampling k in parallel costs k× the tokens and k× the sandbox-minutes to move success from pass@1 toward pass@k.
• Fleet scaling is embarrassingly parallel across independent tasks but bounded by the sandbox runner pool and by the shared model rate limit. The scarce resource is usually sandbox-minutes, not model QPS.

The mature pattern: cap max_iters low (2–4), parallelize samples rather than deepen iterations, cache the verify environment (a warm sandbox with dependencies installed), and reserve the expensive full suite for the final gate while cheap static checks (the lab's ast-based defines_function, lint, type-check) run first to fail fast.

Failure modes and blast radius

Think in blast radius, because the dangerous failures here are quiet correctness and supply-chain regressions, not crashes.

• The path guardrail bypass. The agent edits .github/workflows/deploy.yml, secrets.py, or a dependency manifest. Blast radius: your entire supply chain and CI, because an agent that can edit the pipeline can exfiltrate credentials or ship a backdoor on the next push. The control is an allowed_paths allow-list that fails closed and precedes every write, plus protected paths for CI/secrets that no autonomous run may touch.
• Verify-as-eval RCE. Someone "simplifies" verification to run the model's code in-process. Now agent output is arbitrary code execution with your process's privileges. Blast radius: the host and its credentials. Checks are provided callables; real test runs live in a sandbox.
• Silent file corruption from a fuzzy edit. A tool that replaces "the first match" or regenerates the whole file lands a plausible-but-wrong change that passes a weak suite. Blast radius: every consumer of that file, discovered in production. The control is exact-once-or-fail edits and a suite strong enough to be a real reward.
• The volume trap. The platform is measured on merged-diff throughput. Unreviewed AI code accrues the same debt as unreviewed human code, faster, and a bigger diff is harder to review. Blast radius: the codebase's long-term maintainability and the review team's capacity. The control is the metric itself — verified change, small reviewable diffs — not lines shipped.
• Prompt injection from repo text. The agent reads a ticket, a dependency README, or a doc that carries an injected instruction (Phase 10). It is executing on your side of the trust boundary. Blast radius: anything the guardrail and sandbox do not contain. Defense in depth is the answer: no single layer is trusted alone.

Cross-cutting concerns

Security. Three composing controls, and confusing them is the classic mistake: the allowed_paths allow-list decides what the agent may edit; the sandbox decides what a test run may reach (no network, no ambient creds); human review decides what merges. They fail independently, and a production platform needs all three. Least privilege plus sandbox plus review is the defense-in-depth posture, because the text the agent read might be adversarial.

Cost. The bill is samples × iterations × (tokens + sandbox-minutes). The levers, in order of impact: fail fast on cheap static checks before the full suite; cap iterations and parallelize samples instead of deepening; warm the verify environment; and — the one people skip — a stronger spec reduces iterations more than a bigger model does, because a well-specified task converges faster.

Observability. The transcript is the audit surface: every plan, every apply (with its PatchResult), every verify with its failures. It answers three different questions a dashboard cannot — what did the agent try, why did an edit fail, which check was still red — and it is the artifact a reviewer reads to approve. Treat it as a first-class output, not a debug log.

Multi-tenancy. A shared runner fleet serving many repos needs per-run isolation (a fresh sandbox, a scoped repo checkout, no cross-tenant credential reuse). The repo checkout is the tenant boundary; a leaked path or a shared writable mount is a cross-tenant breach.

The "looks wrong but is intentional" decisions

• Search/replace over unified diff. Edit blocks are more verbose than @@ hunks and look primitive. They are the right call: exact-match fails closed on stale context and refuses on ambiguity, where a context-line diff silently misapplies when the context drifted. Verbosity buys safety and reviewability.
• Static AST checks in the harness instead of running the code. It looks like the verifier is "not really testing." It is deliberate: the harness must never execute agent output. Real execution happens, but behind the sandbox boundary, on purpose.
• A low max_iters. It looks like giving up early. A task that needs more iterations than a small budget is a signal to improve the spec or decompose the task — not to let a confused agent fail more expensively. The cap is a guard, never a target.
• The spec is the eval. Making acceptance checks the contract looks like extra upfront work versus "just tell the agent what you want." It is what gives the loop a stopping condition and turns review from "reading a wall of code" into "confirming the checks are green on a small diff."

Where the coding agent fits the platform decision

A coding agent is one answer to the question every engineering org now faces: how do we let an LLM participate in the SDLC as a first-class actor without trading quality for volume? The framings that ship — Claude Code, Codex, Cursor, Aider — differ in ergonomics and edit format, but the architecture underneath is the same one this phase builds: spec as contract-and-eval, exact-match transactional edits, an objective verify loop bounded by a guard, least-privilege paths, and a human gate on a small verified diff. The principal move is not picking a tool; it is designing the platform so the verified diff is the unit of throughput and the review gate is never optional. Optimize for that, and the tool is a config choice. Optimize for lines shipped, and no tool will save you.