Tool Calling

Phase 10 · Document 01 · Agents and Tools Prev: 00 — Agent Overview · Up: Phase 10 Index

Table of Contents

1. Why This Matters
2. Core Concept
3. Mental Model
4. Hitchhiker's Guide
5. Warmup Readings
6. Deep Readings and External References
7. Key Terms
8. Important Facts
9. Observations from Real Systems
10. Common Misconceptions
11. Engineering Decision Framework
12. Hands-On Lab
13. Verification Questions
14. Takeaways
15. Artifact Checklist

1. Why This Matters

Tool calling (a.k.a. function calling) is the mechanism that makes agents possible — it's the protocol by which a model requests an action and your application executes it and feeds back the result (00). Get the protocol, the schema design, the validation, and the error handling right and your agent is reliable; get them wrong and you get malformed calls, silent failures, infinite loops, and unsafe execution. Because per-step reliability compounds (Phase 5.06), tool-calling correctness is the single biggest lever on whether a multi-step agent succeeds. This is the most mechanical, most foundational doc of Phase 10 — everything else (planning, memory, code/research agents) sits on top of this loop.

2. Core Concept

Plain-English primer: a structured request/response protocol

Tool calling has four moving parts, repeated in a loop (what-happens §1.G, §9):

1. Tool definitions — you send the model a list of tools, each a name + description + JSON-Schema parameters (02). These ride in the request every turn.
2. tool_use (the request) — instead of (or alongside) text, the model emits a structured tool-call block: a tool name, an id, and arguments (JSON matching the schema). This is text shaped like a call — not an action (00 Law 6).
3. Execution (your app) — you parse the block, validate the arguments against the schema, execute the real function (with permissions, 05), and capture the result.
4. tool_result (the response) — you append a result message referencing the same id, and call the model again with the now-larger context so it can decide the next step.

resp = client.messages.create(model=MODEL, system=SYS, tools=TOOLS, messages=messages,
                              max_tokens=2048)
messages.append({"role": "assistant", "content": resp.content})     # record the model's turn
tool_uses = [b for b in resp.content if b.type == "tool_use"]
if resp.stop_reason != "tool_use" or not tool_uses:
    return resp                                                     # final answer
results = []
for tu in tool_uses:                                                # may be SEVERAL (parallel)
    try:
        args = validate(tu.name, tu.input)                          # schema validation [02]
        out  = run_tool(tu.name, args)                              # THE APP executes [05]
        results.append({"type": "tool_result", "tool_use_id": tu.id, "content": out})
    except ToolError as e:                                          # feed errors back, don't crash
        results.append({"type": "tool_result", "tool_use_id": tu.id,
                        "content": f"ERROR: {e}", "is_error": True})
messages.append({"role": "user", "content": results})               # loop again

The id pairing and parallel calls

Each tool_use carries an id; the matching tool_result references it via tool_use_id. This pairing is what lets the model emit multiple tool calls at once (parallel tool calls — e.g., fetch three URLs simultaneously) and correctly match each result. Every tool_use must get a corresponding tool_result (even on error) or the conversation is malformed and the next call fails. (Across providers the field names differ — OpenAI tool_calls/tool_call_id, Anthropic tool_use/tool_use_id — your gateway/adapter normalizes this, Phase 8.03.)

Tool schema quality is reliability

The model decides whether and how to call a tool entirely from its name, description, and parameter schema — so schema quality directly sets call accuracy:

• Clear, specific descriptions (when to use it, what it returns) — the model reads these to choose.
• Strong typing + constraints (enum, integer, minimum/maximum) narrows the model toward valid args (02).
• Mark only truly-required fields required; give optional fields sensible defaults.
• Single responsibility + distinct names — don't make two tools sound alike (the model will confuse them).
• Right granularity — too many fine-grained tools overwhelm selection; too few god-tools force complex args. Most agents do best with a small, well-named toolset.

Validation, errors, and retries (per-step robustness)

The model will sometimes emit invalid args, hallucinate a tool, or call something that fails. Robust loops:

• Validate every call against the schema before executing; on failure, return a structured error as a tool_result so the model can correct itself (don't crash the loop).
• Execute defensively — timeouts, exceptions caught, results truncated/summarized if huge (a 10k-line output bloats context, Phase 9.07).
• Feed errors back, don't hide them — "ERROR: customer_id not found" lets the model retry/adjust; a silent failure makes it loop.
• Bound retries — a few self-corrections, then give up / escalate (ties to stop conditions, 00).

tool_choice and standards (MCP)

• tool_choice controls calling: auto (model decides), required/any (must call some tool), none (text only), or force a specific tool — useful to constrain behavior (e.g., force structured extraction, 02).
• MCP (Model Context Protocol) is an emerging open standard for exposing tools/data to any agent, so you define a tool once and many clients can use it — the "USB-C for tools." Worth knowing as the interoperability direction.

3. Mental Model

   TOOL DEFINITIONS (name + description + JSON-Schema params) ride in every request [02]
        ▼
   model emits tool_use {id, name, args}  ← a REQUEST, not an action [00 Law 6]
        ▼  per call (may be PARALLEL — match by id)
   APP: VALIDATE args [02] → EXECUTE (permissions [05], timeout, truncate) → tool_result {tool_use_id, content/is_error}
        ▼  EVERY tool_use needs a tool_result (even errors)
   call model again with growing context → next tool_use or FINAL answer (stop_reason)

   schema quality = call accuracy · feed ERRORS back (don't crash/hide) · bound retries
   tool_choice: auto / required / none / force-specific   ·   MCP = standard tool exposure

Mnemonic: define tools as schemas; the model emits tool_use (a request), the app validates+executes+returns tool_result (matched by id). Schema quality = reliability; feed errors back; every call gets a result.

4. Hitchhiker's Guide

What to look for first: the loop correctness (every tool_use → a tool_result, matched by id) and schema quality (clear descriptions, strong types). Those two set reliability.

What to ignore at first: MCP, exotic tool_choice tricks, and huge toolsets. Start with a few well-described read-only tools and a clean validate→execute→return loop.

What misleads beginners:

• Auto-executing tool_use. It's a request — validate + permission first (00, 05).
• Vague/overlapping schemas. The model can't pick the right tool or fill args correctly — the #1 cause of bad calls.
• Swallowing errors. A silent failure makes the model retry blindly or loop; return a structured error it can act on.
• Dropping a tool_result. Missing the result for any tool_use malforms the conversation and breaks the next call.
• Dumping huge tool outputs. Unbounded results bloat context (cost + lost-in-the-middle, Phase 9.07) — truncate/summarize.
• Too many tools. Overwhelms selection; prefer a small, distinct set.

How experts reason: they treat tool calling as a typed request/response protocol: small, well-named, strongly-typed tools; validate every call and return structured errors for self-correction; bound retries; truncate large outputs; pair tool_use/tool_result by id; and use tool_choice to constrain when needed. They measure per-step tool-call reliability (09) because it compounds.

What matters in production: valid-call rate, error-recovery behavior, result-size discipline (context cost), id-pairing correctness, and that the app — not the model — gates execution (05).

How to debug/verify: inspect the trace (08): which tool, args, result; was a call invalid (schema), mis-selected (description), or unrecovered (error not fed back)? Replay with a tightened schema or clearer error.

Questions to ask: are schemas clear/typed/distinct? is every call validated before execution? are errors fed back structurally? are results bounded? is the app the execution boundary? does the gateway normalize provider tool formats (Phase 8.03)?

What silently gets expensive/unreliable: vague schemas (mis-calls), swallowed errors (loops), unbounded outputs (context bloat/cost), missing tool_results (broken calls), and auto-executed unvalidated calls (unsafe).

5. Warmup Readings

6. Deep Readings and External References

7. Key Terms

8. Important Facts

• Tool calling is a request/response protocol: tool definitions → tool_use (model's request) → app validates+executes → tool_result (keyed by id) → loop (what-happens §1.G).
• A tool_use is text shaped like a call, not an action — the app is the execution/trust boundary (00 Law 6, 05).
• Every tool_use must get a matching tool_result (even on error), or the conversation is malformed.
• Parallel tool calls are matched to results by id.
• Schema quality = call accuracy — clear descriptions, strong types/enums, distinct names, single responsibility, small toolset.
• Validate before executing; return structured errors so the model self-corrects; bound retries.
• Truncate/summarize large tool outputs — unbounded results bloat context (cost + lost-in-the-middle, Phase 9.07).
• tool_choice constrains calling; MCP is the emerging standard for exposing tools to any agent; gateways normalize provider tool formats (Phase 8.03).

9. Observations from Real Systems

• Every production agent runs this loop — Claude Code, Cursor, and custom agents all emit tool_use and execute app-side (06, Phase 11).
• Parallel tool calls are widely used to fan out reads (e.g., fetch several files/URLs at once) and cut latency (07).
• MCP is being adopted to expose tools/data uniformly so the same tool server works across many agent clients.
• The most common agent bug is a tool-schema problem — vague descriptions or weak types causing mis-selection/invalid args (09).
• Structured error feedback (returning "ERROR: …" as a tool_result) is what lets good agents recover instead of looping (08).

10. Common Misconceptions

11. Engineering Decision Framework

IMPLEMENT TOOL CALLING:
 1. DEFINE small, distinct, strongly-typed tools (clear description + JSON Schema [02]); start READ-ONLY.
 2. LOOP: model → tool_use → VALIDATE args [02] → EXECUTE (app, permissions [05], timeout) → tool_result(id).
      - emit a tool_result for EVERY tool_use (errors as is_error); match parallel calls by id.
 3. ROBUSTNESS: structured errors fed back for self-correction; BOUND retries; TRUNCATE big outputs [9.07].
 4. CONTROL: tool_choice (auto/required/none/specific) to constrain when needed.
 5. SAFETY: app is the execution boundary; risky tools → sandbox/approval [05].
 6. MEASURE per-step valid-call rate + recovery [09]; normalize provider formats at the gateway [8.03].

12. Hands-On Lab

Goal

Implement a robust tool-calling loop and prove that schema quality and error feedback drive per-step reliability.

Prerequisites

• pip install openai (or Anthropic SDK); the bounded loop from 00.

Steps

1. Define two tools with clean schemas (e.g., get_weather(city, units enum), search_db(query, limit int)); send them with tool_choice="auto".
2. Implement the loop: model → tool_use → validate args against the schema → execute (mock) → return tool_result keyed by id → re-call until final. Ensure every tool_use gets a tool_result.
3. Parallel calls: ask something that needs two tools at once; confirm the model emits parallel tool_use blocks and you match each result by id.
4. Schema A/B: run a task with (a) a vague schema ("does stuff", data: string) vs (b) a clear, typed schema; measure valid-call rate over 10 runs — the clear schema should win clearly.
5. Error feedback: make a tool fail (e.g., unknown id); return a structured error (is_error) and show the model self-corrects; then try swallowing the error and show it loops/guesses.
6. Output discipline: return a deliberately huge tool output; add truncation/summarization and show context/cost drop (Phase 9.07).

Expected output

A working tool-calling loop (with parallel-call id matching), a schema-quality valid-call-rate comparison, an error-feedback self-correction demo, and an output-truncation before/after.

Debugging tips

• "tool_result without tool_use" / malformed error → you dropped or mis-id'd a result.
• Low valid-call rate → vague schema or weak typing; tighten descriptions/enums.

Extension task

Add tool_choice to force a specific tool for a structured-extraction step (02); add bounded retries with a final escalation.

Production extension

Route execution through the trust boundary with permissions (05), normalize provider tool formats at the gateway (Phase 8.03), and trace every call (08).

What to measure

Valid-call rate (clear vs vague schema), parallel-call correctness, self-correction on errors, context/cost before/after truncation.

Deliverables

• A robust tool-calling loop (validate → execute → result, id-matched, errors fed back).
• A schema-quality valid-call-rate comparison.
• An error-feedback self-correction demo + output-truncation before/after.

13. Verification Questions

Basic

1. What are the four parts of the tool-calling protocol?
2. What is a tool_use block — an action or a request? Who executes it?
3. Why must every tool_use get a tool_result?

Applied 4. Why does tool-schema quality determine call accuracy? Give three schema rules. 5. How do parallel tool calls get matched to their results?

Debugging 6. The agent keeps picking the wrong tool. Cause and fix. 7. A tool fails and the agent loops forever. What's missing?

System design 8. Design a robust tool-calling layer: validation, structured errors, retries, output limits, trust boundary.

Startup / product 9. Why is per-step tool-call reliability the biggest lever on multi-step agent success?

14. Takeaways

1. Tool calling is a request/response protocol: definitions → tool_use (request) → app validates+executes → tool_result (by id) → loop.
2. The model proposes; the app executes — and every tool_use needs a tool_result (errors included).
3. Schema quality = call reliability — small, distinct, strongly-typed tools with clear descriptions.
4. Validate, feed structured errors back, bound retries, and truncate big outputs for robust, cheap loops.
5. Use tool_choice to constrain; know MCP; normalize provider formats at the gateway (Phase 8.03) — and measure per-step reliability (09).

15. Artifact Checklist

• A robust tool-calling loop (validate → execute → id-matched result, errors fed back).
• A parallel-call correctness demo (id matching).
• A schema-quality valid-call-rate comparison.
• An error-feedback self-correction demo.
• An output-truncation before/after (context/cost).

Up: Phase 10 Index · Next: 02 — JSON Schema and Structured Output