Phase 10 — Agents and Tools
How to turn a stateless text generator into something that acts — safely. The agent loop and tool-calling protocol, structured output, planning architectures, memory, the sandbox/approval safety layer, the two dominant applications (code and research agents), and the observability + evaluation that make agents debuggable and trustworthy.
Why this phase matters
An agent wraps the model in a loop where it proposes actions and your application executes them (what-happens §1.G). It's the highest-leverage and highest-risk LLM pattern: the same loop that automates a workflow can also delete files, leak data, or run away in cost. Two principles govern the whole phase: the model proposes, the application executes (the trust boundary where all safety lives), and per-step reliability compounds (0.95¹⁰ ≈ 0.60), so agent quality is the loop — tools, recovery, stops, safety — not raw model IQ (Phase 5.06). And the seniority signal: use the least-agentic thing that works (workflow before agent).
Documents
| # | Document | What you'll be able to do |
|---|---|---|
| 00 | Agent Overview | The loop, model-proposes/app-executes, agent vs workflow, compounding |
| 01 | Tool Calling | The tool_use/tool_result protocol, validation, retries, parallel calls |
| 02 | JSON Schema & Structured Output | Constrained output — and why constrained ≠ correct |
| 03 | Planner-Executor | ReAct, plan-and-execute, reflection, orchestrator–worker (least-agentic) |
| 04 | Agent Memory | Manage a growing context: trim, compact, scratchpad, retrieve, subagents |
| 05 | Sandbox and Approval | The safety layer: isolation, risk tiers, approval, least privilege, audit |
| 06 | Code-Editing Agents | Retrieve→edit→verify→fix; apply-rate; task-resolution |
| 07 | Browser and Research Agents | Agentic RAG; orchestrator–worker; prompt-injection containment |
| 08 | Agent Observability | Trace the trajectory; per-task metrics; loop detection; replay |
| 09 | Agent Evaluation | Outcome + trajectory + safety; compounding; the regression gate |
How to work through it
Read 00 (the loop + the two governing principles) first. 01–02 are the mechanics (tool calling + structured output) every agent sits on. 03–04 are how the agent organizes work and manages context. 05 is the non-negotiable safety layer — read it before giving any agent write access. 06–07 are the two dominant applications (and 06 bridges to Phase 11). 08–09 are how you see and measure multi-step behavior — built together, since traces are the eval substrate. Every doc opens with a from-zero plain-English primer and ends with a buildable, measured lab. The phase builds directly on the agent-loop and memory references, and composes with serving (Phase 7), gateways (Phase 8), RAG (Phase 9), coding platforms (Phase 11), eval (Phase 12), and security (Phase 14).
Phase 10 artifacts
- A bounded agent (read-only tools + stop conditions) + a workflow-vs-agent comparison (00).
- A robust tool-calling loop (validate→execute→result, errors fed back) + schema-quality A/B (01).
- A conformance-vs-value-accuracy demo (constrained ≠ correct) (02).
- An architecture comparison (ReAct/plan-execute/reflection/multi-agent) (03).
- A managed-memory agent (trim+compact+scratchpad) with cost-per-step (04).
- A risk-tiered, sandboxed, approval-gated, audited agent + injection red-team (05).
- A code-editing agent with apply-rate + task-resolution (06); a research agent with citations + injection containment (07).
- A traced agent with per-task metrics + loop detection (08); an eval harness (outcome+trajectory+safety) + regression gate (09).