« Phase 02 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 02 Warmup — Tool Calling & Structured Output

Who this is for: you did Phases 00–01 and can write Python. You've maybe called a tool-using model through an SDK but never built the layer that validates and executes what it emits. By the end you'll have built a JSON-Schema validator, a native-tool-call parser, a validate-before-execute dispatcher, and a repair loop — the machinery every agent framework hides. No real model; the "model" is injected text.

What tool calling actually is
The wire format: schemas in, tool calls out
JSON Schema as the contract
Building a validator from scratch
Validate before you execute
The double-encoded-arguments gotcha
JSON mode vs tool mode vs constrained decoding
Constrained ≠ correct: the repair loop
Structured output beyond tools
Tool design: the part that decides reliability
Parallel tool calls
Common misconceptions
Lab walkthrough
Success criteria
Interview Q&A
References

1. What tool calling actually is

A language model can only read and write text. It cannot search the web, query a database, or send an email. Tool calling (also "function calling") is the bridge: you describe some functions to the model, and when it wants one run, it emits a structured request naming the function and its arguments; your program parses that request, runs the real function, and feeds the result back. This is the concrete implementation of Phase 00's trust boundary — the model proposes a call; your application executes it — and it's the single mechanism by which every agent affects the world.

The critical mental shift from Phase 01: the model does not run the tool. It never has and never will. It produces a description of a call. Everything that makes that call safe, correct, and authorized happens in your code, after the model is done talking. That is why this phase is about validation, not about the model.

2. The wire format: schemas in, tool calls out

The protocol has two directions:

Into the model you send a list of tool definitions, each a name, a natural-language description (the model reads this to decide when to use the tool), and a JSON Schema for the parameters. In OpenAI's format that's {"type": "function", "function": {"name", "description", "parameters": <schema>}}; Anthropic's is {"name", "description", "input_schema"}. Same idea, cosmetic differences.

Out of the model you get one or more tool calls: {"name": "search", "arguments": {"q": "..."}}. The model chose the tool and filled the arguments; your job is to parse, validate, dispatch, and return the result as a tool message so the loop (Phase 01) continues.

The description matters more than beginners expect: it is the model's only guidance on when and how to call the tool. A vague description ("does stuff with orders") produces wrong calls; a precise one ("Look up an order by its numeric ID; returns status and total. Use only when the user gives an explicit order number.") produces right ones. You are writing a tiny spec for a probabilistic caller.

3. JSON Schema as the contract

JSON Schema is a standard, machine-checkable way to describe the shape of JSON data. It is the contract between the model's proposal and your executor. The keywords you'll use constantly:

type — "object", "array", "string", "integer", "number", "boolean", "null".
properties — the fields of an object and each field's sub-schema.
required — which properties must be present.
enum — the allowed values (e.g. "units": {"enum": ["metric", "imperial"]}).
minimum / maximum, minLength / maxLength — range/length bounds.
items — the sub-schema every array element must match.
additionalProperties: false — reject unknown fields (tighten the contract).

A tool's parameters schema is exactly this. So is a structured output schema (§9). Learning to read and write JSON Schema fluently is a genuine interview asset, because it's the lingua franca of every tool-using system — OpenAI, Anthropic, MCP (Phase 03), Pydantic, and the constrained-decoding libraries all speak it.

Integer vs boolean gotcha. In JSON (and Python) true/false are technically not integers, but a naive isinstance(x, int) check passes True as 1 because bool subclasses int in Python. A correct validator rejects a bool where an integer is required. Your lab's validator handles this — interviewers love the edge.

4. Building a validator from scratch

You will not pip install jsonschema — you'll write validate(instance, schema) returning a list of human-readable error strings (empty = valid). Writing it teaches you what the keywords mean and produces error messages a model can act on (which the repair loop needs). The core is a recursive walk:

def validate(instance, schema, path="$"):
    errors = []
    t = schema.get("type")
    if t == "object":
        if not isinstance(instance, dict):
            return [f"{path}: expected object"]
        for r in schema.get("required", []):
            if r not in instance:
                errors.append(f"{path}.{r}: required")
        for k, subschema in schema.get("properties", {}).items():
            if k in instance:
                errors += validate(instance[k], subschema, f"{path}.{k}")
        if schema.get("additionalProperties") is False:
            for k in instance:
                if k not in schema.get("properties", {}):
                    errors.append(f"{path}.{k}: unexpected property")
    elif t == "integer":
        if isinstance(instance, bool) or not isinstance(instance, int):
            errors.append(f"{path}: expected integer")
    # ... enum, minimum/maximum, arrays via items, etc.
    return errors

The important design choice: collect all errors, don't fail on the first. A model repairing its output does far better with the full list ("amount must be integer; reason is required") than with one error at a time. That is a validation-UX decision that directly raises repair success.

5. Validate before you execute

The single most important line in an agent's tool layer: validate the arguments against the schema before the function runs. If you run first and let the function blow up on bad input, you get a cryptic Python traceback, a half-completed side effect, and no clean way to tell the model what went wrong. If you validate first, a bad call becomes a structured error returned as an observation (Phase 01), the side effect never happens, and the model gets an actionable message.

In the lab, dispatch(registry, call) validates, and on failure returns ToolResult(ok=False, error=...) — it never raises. This is the reliability lever from Phase 00 §5 applied to tools: a validated-and-rejected call is a recoverable event, not a failure, so end-to-end reliability goes up. It's also the security lever: validation is your first line against a malformed or adversarial call (Phase 10) reaching real code.

6. The double-encoded-arguments gotcha

Here's a bug that has cost real teams real hours. Many providers return arguments not as a JSON object but as a JSON string — a string whose contents are themselves JSON:

{"name": "search", "arguments": "{\"q\": \"tokyo\"}"}

arguments is the string "{\"q\": \"tokyo\"}", and you must json.loads it a second time to get the dict. If you treat it as already-parsed, everything downstream sees a string where it expects an object, and validation fails confusingly. Your lab's parse_tool_call handles both shapes (object or double-encoded string) — a small robustness detail that separates code that works in the demo from code that works against three providers.

7. JSON mode vs tool mode vs constrained decoding

There are three ways to get structured output from a model, and knowing the difference is a common interview probe:

Tool / function mode — you pass tool schemas; the model returns tool calls. Best when the model should choose whether and which tool to call.
JSON mode — you ask the model to reply with JSON (some providers accept a response schema). Best when you always want one structured answer, not a choice of tools.
Constrained / grammar-guided decoding — libraries like outlines, guidance, or llama.cpp's GBNF grammars mask the model's output logits at each step so only tokens that keep the output valid against a grammar/schema are allowed. This guarantees the output parses — it is syntactically valid by construction.

The trap is thinking constrained decoding solves correctness. It guarantees the JSON is well-formed, not that the values are right. A grammar can force {"amount": <integer>} but cannot force the correct integer. Which is why…

8. Constrained ≠ correct: the repair loop

…you still validate semantically and repair. The repair loop is: validate the model's tool call; if invalid, feed the specific errors back to the model ("amount must be a positive integer; you sent -5") and ask it to re-emit; re-validate; bounded by a max_repairs count so a model that can't get it right doesn't loop forever. In the lab, run_with_repair(fixer_policy, registry, call, max_repairs=2) implements exactly this, with the model injected as a pure fixer_policy(errors, call) -> corrected_call.

Two design points:

Bound it. Repairs cost tokens and latency, and a model stuck on a genuinely impossible call (missing data it doesn't have) will never succeed. Cap the attempts and surface the failure to a human or a fallback.
The error message is the whole game. The model can only fix what you tell it is wrong. Precise, structured errors → high repair success; vague ones → the model guesses again. This is why §4's "collect all errors" matters.

"Constrained ≠ correct" — say it in the interview and you've signaled you understand the difference between syntactic and semantic validity, which most candidates conflate.

9. Structured output beyond tools

Tool calling is one use of schemas; structured output is the other. Often you don't want the agent to act — you want it to return data in a fixed shape (extract fields from a document, classify with a rationale, produce a typed record). Same machinery: define an output JSON Schema, get the model's JSON, validate, repair. The lab's coerce_output(text, schema) does this, including the classic robustness detail that models often wrap JSON in ```json fences you must strip before parsing.

This is the backbone of libraries like Pydantic AI and instructor: you declare a Pydantic model (which compiles to a JSON Schema), the library gets the model to emit conforming JSON, and you get a typed Python object back. You just built the mechanism underneath.

10. Tool design: the part that decides reliability

The biggest lever on agent reliability isn't the loop — it's the tools. Principles that show up in every good agent codebase and every senior interview:

Few, well-scoped tools beat many overlapping ones. A model faced with 40 similar tools picks wrong; 6 clear ones it uses correctly. (This raises per-step reliability, which compounds — Phase 00.)
Names and descriptions are prompts. get_order_by_id with "Look up an order by numeric ID" beats orders with "order stuff." The description is the model's spec.
Make errors model-actionable. Return "order 12345 not found; check the ID" not "KeyError". The model reads your error and recovers.
Design for the model's blind spots. Don't require the model to compute or remember things it's bad at — pass IDs, not free text it must reconstruct; return enums, not prose it must re-parse.
Idempotency where possible. A tool that's safe to call twice (Phase 00/08) can be retried, which buys reliability.

This is unglamorous and it is the differentiator between an agent that works 60% of the time and one that works 95%.

11. Parallel tool calls

Modern models can request several tool calls in one turn when the calls are independent (e.g. "look up the weather in three cities"). Your executor should run them concurrently and return all results together — a latency win (the calls are a max, not a sum, like ReWOO's parallel steps in Phase 01). The gotcha: only parallelize genuinely independent calls; if call B needs call A's result, they're sequential, and a model that emits them in parallel has made an error your orchestration should catch.

12. Common misconceptions

"The model runs the tool." It emits a description of a call; your code runs it. All safety/correctness is on your side.
"Constrained decoding means the output is correct." It means it parses. Values can still be wrong; validate and repair.
"isinstance(x, int) is a fine integer check." In Python it passes True. Reject bools.
"arguments is always a dict." Often it's a JSON string you must parse again.
"More tools = more capable agent." More tools = more wrong choices. Fewer, sharper tools win.
"Validation is optional if the prompt is good." The prompt is a suggestion to a stochastic generator. Validation is the control.

13. Lab walkthrough

Open lab-01-tool-calling-engine/ and fill the TODOs:

validate(instance, schema) — the recursive validator (object/array/scalars, required, enum, ranges, additionalProperties, bool-not-int). Collect all errors. This is the heart.
Tool / ToolRegistry — hold the schema; to_openai_schema() emits what a model receives.
parse_tool_call(raw) — handle both a dict and a double-encoded arguments string; malformed → ValueError.
dispatch — validate before running; structured error on failure, never raise.
run_with_repair — the bounded repair loop with an injected fixer_policy.
coerce_output — structured output incl. stripping ```json fences.

Run LAB_MODULE=solution pytest -v first to see the target, then match it. Read solution.py's main() — it shows a valid call, a rejected one, a repair in one round, and fenced-JSON coercion.

14. Success criteria

You can write a JSON Schema for a 3-argument tool and say what each keyword guarantees.
Your validator rejects a bool where an integer is required and collects all errors.
Your dispatch validates before executing and returns a structured error, never raising.
You can explain "constrained ≠ correct" and how the repair loop addresses it.
All 30 tests pass under lab and solution.

15. Interview Q&A

Q: How does tool calling actually work end to end? A: You send the model tool definitions (name, description, JSON-Schema parameters). It returns a structured call {"name", "arguments"}. Your code parses it, validates the arguments against the schema before executing, runs the real function, and returns the result as a tool message so the loop continues. The model only proposes; your code executes and validates — that's the trust boundary in code.

Q: A model emits valid JSON but the wrong value. Where does that get caught? A: Not by constrained decoding — that only guarantees well-formed JSON. It's caught by semantic validation: schema checks (types, enums, ranges) plus business rules, and then a bounded repair loop that feeds the specific error back for a fix. "Constrained ≠ correct."

Q: Why validate before executing instead of catching exceptions? A: Validating first means the side effect never happens on a bad call, the failure is a structured, model-actionable observation rather than a cryptic traceback, and it's the first security gate against malformed or adversarial arguments reaching real code. Catching after means you've already half-run the tool.

Q: How do you design tools for reliability? A: Few well-scoped tools over many overlapping ones; precise names and descriptions (they're the model's spec); model-actionable error messages; pass IDs/enums rather than free text the model must reconstruct; make tools idempotent so they're safe to retry. Tool design moves per-step reliability more than the loop does, and reliability compounds.

Q: What's the double-encoded arguments gotcha? A: Some providers return arguments as a JSON string, not an object, so you must json.loads it twice. Miss it and everything downstream sees a string where it expects a dict, and validation fails confusingly.

Q: When would you use JSON/response mode vs tool mode vs constrained decoding? A: Tool mode when the model should choose whether/which tool to call; JSON/response mode when you always want one structured answer; constrained/grammar decoding (outlines/guidance/GBNF) when you need a hard guarantee the output parses. All three still need semantic validation.

16. References

JSON Schema specification. https://json-schema.org/
OpenAI — Function calling guide. https://platform.openai.com/docs/guides/function-calling
Anthropic — Tool use (function calling). https://docs.anthropic.com/en/docs/build-with-claude/tool-use
outlines (constrained/structured generation). https://github.com/dottxt-ai/outlines
guidance (constrained decoding). https://github.com/guidance-ai/guidance
Pydantic AI. https://ai.pydantic.dev/ · instructor. https://python.useinstructor.com/
Willard & Louf, Efficient Guided Generation for LLMs (2023). https://arxiv.org/abs/2307.09702
llama.cpp GBNF grammars. https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md

Agentic AI Engineer — Agent Infrastructure, Platforms & Production Systems