« Phase 01 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 01 Warmup — The Agent Loop From Scratch

Who this is for: you can write Python and you did Phase 00. You have never built the loop that turns an LLM into an agent. By the end you will have built three of them and know exactly why each exists. No framework, no API key — the model is a function you inject.

The loop, stated precisely
Why we inject the model
ReAct: interleaving reasoning and acting
Parsing a model step across the trust boundary
Errors as observations, not exceptions
The step budget: why max_steps is non-negotiable
ReWOO: plan, execute without the model, solve
Variable substitution: wiring outputs into inputs
Plan-execute-replan: buying back adaptivity
Choosing a mode: the adaptivity–cost axis
Common misconceptions
Lab walkthrough
Success criteria
Interview Q&A
References

1. The loop, stated precisely

An agent is a loop. Here it is with nothing hidden:

scratchpad := task
repeat up to max_steps times:
    proposal := model(scratchpad)          # UNTRUSTED text
    if proposal is a final answer:  return the answer
    (name, args) := parse(proposal)        # cross the trust boundary
    observation := execute(name, args)     # run the tool (or observe its error)
    scratchpad := scratchpad + proposal + observation   # remember
return "gave up" (the max_steps guard fired)

Five things are doing all the work, and each becomes a later phase: the model (untrusted oracle), parse (the trust boundary, Phase 02), execute (the sandbox and authz, Phases 09/13), the growing scratchpad (memory and cost, Phase 04), and max_steps (reliability, Phase 00). This warmup builds the loop three ways; the rest of the track hardens each piece.

2. Why we inject the model

The LLM is the only non-deterministic part of an agent. If we call a real one in a test, the test is flaky, slow, and costs money — and worse, it tests the model, not our code. So in every lab we replace the model with an injected policy: a plain function Policy: str -> str that takes the prompt and returns the "model's" text. In tests the policy is a scripted, pure function of its input, so the whole loop is reproducible and we can assert the exact trace.

This is not a toy shortcut. It is exactly how production teams unit-test agents — record/replay fixtures, FakeLLM stubs, VCR-style cassettes. And it enforces the deepest discipline of the craft: your loop's correctness must not depend on the model being right. The loop must survive a wrong tool call, a malformed step, an infinite-loop policy. If it only works when the model behaves, it is not production code. The injected policy makes you prove that.

3. ReAct: interleaving reasoning and acting

ReAct (Yao et al., 2022) is the canonical loop. The model, each turn, writes a short Thought (its reasoning), an Action (a tool name), and an Action Input (the arguments); your code runs the tool and feeds back an Observation; repeat. A run looks like:

Thought: I should look up France.
Action: search
Action Input: {"q": "france"}
Observation: Paris is the capital of France
Thought: Now I can answer.
Final Answer: Paris

The word "synergy" in the paper's title is the point: reasoning guides which action to take, and the action's observation grounds the next reasoning. The model sees every observation, so it is maximally adaptive — if a search returns something surprising, the next Thought can react to it. That adaptivity is ReAct's strength.

Its cost is structural. Every step, the entire scratchpad so far is the prompt (the model needs the history to reason well), so input tokens grow quadratically (Phase 00 §6), and there is exactly one LLM call per step. In the lab, run_react records llm_calls == steps — a test asserts it, so the cost model stops being abstract.

4. Parsing a model step across the trust boundary

The model emits text. Your code needs structure — a tool name and typed arguments — to do anything with it. parse_react_step is that conversion, and it is where the trust boundary (Phase 00 §2) is physically crossed: untrusted text in, structured proposal out.

Two rules make parsing production-grade:

Be liberal in what you accept, strict in what you produce. Models are sloppy — extra whitespace, a missing Thought, a code fence around the JSON. The parser tolerates the cosmetic and extracts the essentials (regex for Action: / Action Input: / Final Answer:, then json.loads the arguments).
Malformed input is data, not a crash. If the model emits garbage — no Action, non-JSON arguments — parse_react_step raises ValueError, and the loop catches it and records a PARSE_ERROR observation. A real agent would feed that error back to the model ("your last step was malformed, try again"). The run degrades gracefully; it does not explode.

Production note. Modern models emit native tool calls — structured JSON in a dedicated field, not free text you regex. That is strictly better (Phase 02 covers it), but the ReAct text format is still what you'll see in traces, prompts, and older stacks, and parsing it teaches the boundary discipline that the native path also needs (you still validate the JSON against a schema).

5. Errors as observations, not exceptions

Here is a rule that separates toy agents from real ones: a tool failure is an observation the agent recovers from, not an exception that kills the run. In the lab, ToolRegistry.dispatch never raises — an unknown tool, wrong arguments, or an exception inside the tool all become a returned ToolError whose as_observation() is fed back into the loop:

Action: divide
Action Input: {"x": 10, "y": 0}
Observation: ERROR[divide]: ZeroDivisionError: division by zero
Thought: I can't divide by zero; let me try another approach.

Why does this matter so much? Because in production, tools fail constantly — a flaky API, a rate limit, a bad argument the model hallucinated. If any of those crash the agent, your reliability is capped by your least-reliable tool. By making failures observable, the agent can retry, route around, or ask for help — and your 0.95^n per-step reliability (Phase 00) goes up, because a recoverable failure isn't a failure. This is the same principle as returning an error body from an HTTP API instead of hanging up the connection.

6. The step budget: why max_steps is non-negotiable

An LLM can get confused and propose the same action forever. Without a hard cap, that is an infinite loop burning tokens and never terminating — a production incident. max_steps is the guard: the loop runs at most that many times, and hitting the cap returns a partial trace marked max_steps, not a hang.

max_steps is a safety guard, not a target (Phase 00 §5). You set it from your reliable step budget plus a small margin. If a task legitimately needs more steps than the budget allows, that's a signal to add verification or decomposition — not to raise the cap, which just lets a confused agent fail more expensively. The lab tests this directly: a policy that loops forever stops at exactly max_steps with the right stopped_reason.

7. ReWOO: plan, execute without the model, solve

ReWOO (Xu et al., 2023) — Reasoning WithOut Observation — restructures the loop to kill the quadratic. Instead of interleaving, it has three modules:

Planner — one LLM call that writes the entire plan as a list of steps, each naming a tool and its arguments, using variable references (#E1, #E2) so a later step can consume an earlier step's result. Crucially, the planner writes this before any tool runs — it reasons "without observation."
Workers — your code runs each tool in order, filling in the evidence variables. The LLM is not in this loop. No re-reading a scratchpad, no per-step model call.
Solver — one final LLM call that reads the task plus all gathered evidence and writes the answer.

So the model is called exactly twice, no matter how many tools run. The lab's run_rewoo records llm_calls == 2 and a test asserts it against a 3-tool plan — the linear-vs-quadratic win from Phase 00, now visible in the trace. Because the tool steps are decoupled from the model, independent ones can even run in parallel (a further latency win; the lab suggests it as an extension).

The price, again, is adaptivity: the planner commits before seeing any observation, so if step 2's result invalidates the plan, ReWOO barrels on and the Solver inherits the mess. ReWOO is the right call when the plan is fairly predictable and cost/latency matter.

8. Variable substitution: wiring outputs into inputs

The mechanism that lets ReWOO run tools with the LLM out of the loop is variable substitution. A plan step's arguments may contain references like "#E1"; before running the step, your code replaces each reference with the actual result of the earlier step. Example plan (JSON):

{"plan": [
  {"id": "E1", "tool": "search",     "args": {"q": "france"}},
  {"id": "E2", "tool": "word_count", "args": {"text": "#E1"}},
  {"id": "E3", "tool": "multiply",   "args": {"x": "#E2", "y": 2}}
]}

E2 consumes E1's output; E3 consumes E2's. The one subtlety the lab makes you get right: preserve type on an exact reference. If the whole argument value is exactly "#E2" and E2 produced the integer 6, the substitution must pass the integer 6 to multiply, not the string "6" — otherwise multiply gets a string and fails. If #E1 appears inside a larger string, you interpolate str(value). An unresolved reference (#E9 with no E9) is a plan error the caller must handle. This tiny function is the whole reason ReWOO can keep the model out of the execution loop.

9. Plan-execute-replan: buying back adaptivity

ReWOO's weakness is that it can't react to a failure. Plan-execute-replan fixes exactly that while keeping the cost advantage. It plans (1 call), executes step by step, and if a step fails — a tool errors, a reference won't resolve — it re-plans: it calls the planner again, this time with the failure and the evidence-gathered-so-far in the context, so the planner can route around the broken step. The number of replans is bounded (max_replans) so a 永-failing task terminates instead of looping.

In the lab, run_plan_execute records llm_calls == 1 + replans: a task that succeeds on the first plan costs one call; one that needs a single recovery costs two. Compare that to ReAct, which would pay a call every step regardless. This is the reliability lever from Phase 00 §5 (verify/recover) implemented as control flow, and it's what most production "planner" agents (LangGraph plan-execute with a re-plan edge, ADK's LoopAgent around a planner) actually do.

10. Choosing a mode: the adaptivity–cost axis

The three modes are three points on one axis:

  more adaptive, more expensive  ◄─────────────────────►  cheaper, less adaptive
        ReAct              plan-execute-replan                 ReWOO
   (LLM call/step)         (LLM call + replans)          (2 LLM calls total)
   uncertain path,         mostly-known path w/          predictable path,
   needs mid-course        occasional failure to         cost/latency matter,
   correction              recover from                  steps parallelizable

You pick per task, and real frameworks let you mix: a LangGraph graph can have a planning node feeding interleaved worker nodes; ADK composes Sequential, Parallel, and Loop workflow agents around LlmAgents. The lab's run facade is the miniature of that: one entry point, a mode argument, the right loop underneath. The skill is not memorizing a framework's node types — it's knowing, from Phase 00's numbers and this phase's mechanics, which shape a task wants.

11. Common misconceptions

"ReAct is the 'real' agent and ReWOO is a hack." Both are legitimate; they trade adaptivity for cost. The senior move is choosing, not defaulting.
"A tool error should raise." No — it should be an observation the agent can recover from. Raising caps your reliability at your flakiest tool.
"Higher max_steps = more capable agent." Higher max_steps = a confused agent burns more tokens before failing. It's a guard, not a capability dial.
"The planner in ReWOO sees the observations." It does not — that's the whole point of "WithOut Observation." Only the Solver sees them, once.
"Substitution is just string replace." Exact references must preserve type, or the next tool gets a string where it wanted a number. That bug is a classic.
"Frameworks make this obsolete." Frameworks make it ergonomic. When one loops or picks the wrong tool at 2 a.m., the person who built the loop from scratch is the one who fixes it.

12. Lab walkthrough

Open lab-01-multimode-agent-runtime/ and fill the TODOs top to bottom:

ToolRegistry.dispatch — the unknown-tool guard and the try/except that turns any exception into a ToolError. Get this right first; the whole loop depends on it.
parse_react_step then run_react — the interleaved loop, the max_steps guard, the parse-error path. Confirm llm_calls == steps.
parse_plan and substitute — the JSON plan parser and the type-preserving reference resolver. The substitution edge cases are where the tests bite.
run_rewoo — two LLM calls, evidence chaining, error paths. Confirm llm_calls == 2.
run_plan_execute — execute, detect failure, rebuild the context with "FAILED at …", replan, bound with max_replans.
run — the mode facade.

Run LAB_MODULE=solution pytest -v first to see green, then match it. Finish by reading solution.py's main() — it runs the same task in all three modes so you can compare the llm_calls counts side by side.

13. Success criteria

You can hand-trace why ReAct records N llm_calls and ReWOO records 2.
Your dispatch never raises; every failure is a ToolError observation.
substitute preserves an int for an exact reference and raises on an unresolved one.
run_plan_execute recovers via replan and terminates at max_replans.
You can state, for a new task, which mode you'd pick and why.
All 18 tests pass under lab and solution.

14. Interview Q&A

Q: Walk me through the agent loop. A: Seed a scratchpad with the task; loop up to max_steps: call the model with the scratchpad, parse its text into either a final answer or a tool call, run the tool (turning any failure into an observation, not a crash), append the observation to the scratchpad, repeat. Stop on a final answer or the step guard. The guards (max_steps), the boundary (parse), and error-as-observation are what make it production code, not the loop itself.

Q: ReAct vs ReWOO — when and why? A: ReAct interleaves, so it's adaptive but pays one LLM call per step and resends the whole scratchpad (quadratic tokens). ReWOO plans once and solves once — two calls total, linear tokens, parallelizable tool steps — but it commits to the plan before seeing any observation, so it can't course-correct. Uncertain path → ReAct; predictable path where cost/latency matter → ReWOO; mostly-known path with occasional failures → plan-execute-replan.

Q: A tool your agent calls starts failing intermittently. What happens to the run, and what should happen? A: If failures are exceptions, the run crashes and reliability is capped by that tool. The right design returns the failure as a structured observation so the agent can retry (if idempotent — Phase 08), route around it, or escalate. That turns a hard failure into a recoverable one and lifts end-to-end reliability.

Q: Why do you inject the model in tests instead of calling a real one? A: Determinism and scope. A real model makes tests flaky, slow, and expensive, and it tests the model, not my loop. Injecting a scripted policy lets me assert the exact trace and, more importantly, prove my loop survives wrong tool calls, malformed steps, and infinite-loop policies — correctness that doesn't depend on the model being right, which is the whole point of production agent code.

Q: How does plan-execute-replan avoid an infinite replan loop? A: max_replans bounds it. Each failure triggers at most one more plan, with the failure in context; once the budget is exhausted the run stops with stopped_reason == "max_replans" and a partial trace, rather than re-planning forever.

15. References

Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022). https://arxiv.org/abs/2210.03629
Xu et al., ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models (2023). https://arxiv.org/abs/2305.18323
Kim et al., An LLM Compiler for Parallel Function Calling (LLMCompiler, 2023). https://arxiv.org/abs/2312.04511
Anthropic, Building Effective Agents (2024). https://www.anthropic.com/research/building-effective-agents
LangGraph — plan-and-execute & ReAct templates. https://langchain-ai.github.io/langgraph/tutorials/plan-and-execute/plan-and-execute/
OpenAI Agents SDK — the Runner agent loop. https://openai.github.io/openai-agents-python/
Google ADK — LlmAgent + Sequential/Parallel/Loop workflow agents. https://adk.dev/

Agentic AI Engineer — Agent Infrastructure, Platforms & Production Systems