Lab 01 — Coding-Agent Harness (Spec → Plan → Apply-Patch → Verify)

Phase 15 · Lab 01 · Phase README · Warmup

The problem

A coding agent — Claude Code, Codex, Cursor, Aider — is not autocomplete. It is the ReAct loop from Phase 01 pointed at a codebase: it reads the repo, plans an edit, applies it as a structured patch, verifies by running the acceptance checks (tests), and — if verification fails — feeds the failures back and tries again. The reward signal is not "the model sounded confident"; it is "the checks pass." That is Spec-Driven Development: the spec plus its acceptance checks are the contract (what to build) and the eval (how you know it's built).

Build a runnable, deterministic model of that loop over an in-memory repo (a dict of path → text — no real filesystem, no subprocess, no exec of model-written code). The model is injected as pure planner/patcher policies, so the whole loop is reproducible and a test can assert the exact iteration count.

What you build

File map

Run it

pytest test_lab.py -v                       # your lab.py (red until you implement)
LAB_MODULE=solution pytest test_lab.py -v   # the reference (green)
python solution.py                          # the worked example

Success criteria

• apply_patch replaces an exactly-once match, and fails without writing when old is missing or ambiguous (appears more than once) — you can explain why guessing is worse than failing.
• A failed apply leaves the repo byte-for-byte unchanged (snapshot/restore rollback).
• verify turns a spec's acceptance checks into a pass/fail with a failures list, and a check that raises becomes a failure, not a crash.
• CodingAgent.run gets the greet spec wrong on iteration 1, sees the failure, and passes on iteration 2 — and stops at max_iters when it never passes.
• The allowed_paths guardrail rejects an out-of-scope edit before any write.
• A reusable command expands to a Spec and runs end-to-end; a missing param is a structured error.
• All 30 tests pass under both lab and solution.

How this maps to the real stack

• apply_patch search/replace is exactly Aider's and Claude Code's edit-block format — a fenced old/new pair the tool locates in the file and swaps. The exact-match, fail-on- ambiguity rule is why those edits are safe and reviewable: the tool never guesses. Codex/GPT and some Cursor paths use a unified diff (@@ hunks with line context); the whole-file form is the fallback for a new file or a heavy rewrite. Same three families, same tradeoffs.
• Spec + verify is Spec-Driven Development — write the verifiable spec first, let the acceptance checks be the contract and the eval. GitHub's Spec-Kit (specify) and the AGENTS.md/CLAUDE.md convention formalize this: a spec/plan file the agent reads and a checklist it must satisfy.
• CodingAgent.run is the loop inside Claude Code / Codex / Cursor: gather context → plan → edit via apply-patch → run tests → iterate. In production the acceptance check is the real test suite run in a sandbox; SWE-bench is this exact loop scored on 2,294 real GitHub issues, where a patch is accepted only if the repo's tests turn green.
• CommandRegistry is Claude Code slash-commands / skills / subagents and Cursor rules — parameterized reusable workflows checked into the repo (.claude/commands/), the "reusable commands / reusable agent workflows" resume language.
• The allowed_paths guardrail is the least-privilege boundary from Phase 09: the agent may edit only the files in scope, and its output is verified, never executed by the checks.

Limits. The acceptance checks here are static predicates over file text/AST because the lab may not exec model-written code (offline, safe, deterministic — the Phase 09 discipline). A real harness runs the actual test suite in a sandbox; a real planner/patcher is an LLM whose plan and diff quality vary; real edits must handle fuzzy context matching, overlapping hunks, and concurrent file state. The control flow — plan, patch, verify, iterate on failure, bound the loop, review the diff — is exactly this.

Extensions (your own machine)

• Add a unified-diff patch shape (@@ -a,b +c,d @@ with context lines) alongside the search/replace and full-file forms; make it fail closed when the context doesn't match.
• Swap the static checks for a real pytest run inside a sandbox (Phase 09) and score a tiny repo the way SWE-bench does — patch accepted only if tests pass.
• Wire a real model behind the planner/patcher interface (one function swap) and add a human-in-the-loop gate: print the diff, require approval before apply_patch writes.
• Add pass@k: run the loop k times with different scripted patchers and report the fraction of specs solved — the agentic-coding eval metric.

Interview / resume signal

"Built a coding-agent harness — a spec → plan → apply-patch → verify loop over an in-memory repo with an injected, deterministic model. Edits are exact search/replace hunks that fail closed on a missing or ambiguous match (Aider/Claude-Code edit blocks) with snapshot/restore rollback; the spec's acceptance checks are the contract and the eval (Spec-Driven Development); a least-privilege allowed_paths guardrail bounds what the agent can touch; and a command/skill registry expands reusable, parameterized workflows into specs. The loop fixes a first-failing edit on the second iteration and is bounded by max_iters — verification, not volume, is the point."