« Phase 21 README · Warmup · Hitchhiker's · Deep Dive · Core Contributor · Staff Notes

Phase 21 — Principal Deep Dive: the OpenAI Agents SDK

The SDK is a beautiful in-process library and a lousy production runtime, and both of those facts are on purpose. This document is about the gap between "the loop runs on my laptop" and "the loop runs the customer-support tier for a company," and about the decisions you make to cross it. If the Deep Dive is the mechanism, this is the architecture that has to survive contact with real traffic, real money, and real incidents.

What the SDK deliberately is not

The Runner loop keeps its entire state — the item list, current agent, turn count — in local variables on one Python stack. That is the correct default for the 95% case (a chat request that starts and finishes inside one HTTP handler) and it is why the SDK is small enough to read in an afternoon. But it means the unit of durability is the process. Crash the process — a deploy, an OOM, a spot-instance reclaim — and the run is gone. There is no checkpoint, no resume, no partial credit. For a 3-second chat, who cares; you retry the whole thing. For a 12-minute research agent that made nine tool calls and two handoffs, "retry the whole thing" means re-paying for all of it and hoping the ninth non-idempotent tool call (charge a card, send an email) doesn't fire twice.

This is the durability gap, and recognizing it as a gap rather than a bug is the first principal-level move. The SDK didn't forget durability; it factored it out, so you can bring your own.

The Temporal wrap: closing the gap without changing the loop

The production answer — the one Temporal's AI Foundations team ships as a named integration — is to run the same loop as a durable workflow. The mapping is clean because the loop is already factored the right way:

• Each model call and each tool call becomes a Temporal activity: a unit that can fail, retry with backoff, and time out independently. Activities are where the nondeterministic, I/O-bound, expensive work lives.
• The loop itself becomes a workflow: an event-sourced function whose every step (activity result, handoff, guardrail decision) is written to a persistent history.

On a crash, Temporal replays the workflow history to reconstruct the exact state — same item list, same current agent, same turn count — and resumes at the next uncompleted activity. Completed model calls and tool calls are not re-executed; their results are read from history. That is the whole payoff: a crash costs you the in-flight activity, not the run.

The constraint that makes this work is the one the entire track has been drilling: the workflow must be deterministic. Replay only reconstructs state correctly if the loop, re-run over the recorded history, makes identical decisions. Read the wall clock, call uuid4(), or iterate a set in hash order inside the workflow body and replay diverges — Temporal detects the nondeterminism and fails the run. This is why every lab in this phase injects the model as a pure policy and uses monotonic counters instead of time.time(). The discipline that makes the labs testable offline is the same discipline that makes the loop replay-safe. That is not a coincidence the curriculum stumbled into; determinism is the shared substrate of "unit-testable" and "durable."

The cost curve, with the math

Guardrails look like a safety feature. Architecturally they are a cost-shaping feature, and the asymmetry is quantifiable. Say the smart agent costs roughly 10 units per run and a fast guardrail model costs 0.1 units. If a fraction p of traffic is abusive/off-topic and an input guardrail catches it before the agent runs, your expected cost per request goes from 10 to 0.1 + (1 - p)·10. At p = 0.2 that is 0.1 + 8 = 8.1 versus 10 — a ~19% cut — and you also dodged the incident that the 20% would have caused. The guardrail pays for itself as long as its own cost is small relative to what it prevents. Put the guardrail on the expensive model "to be accurate" and you invert the math: now every request pays 10 + 10, and you have doubled your bill to save nothing. The guardrail's whole reason to exist is that it sits on the cheap side of the curve.

There is a latency subtlety the real SDK handles that the lab simplifies. In production, input guardrails run concurrently with the first model call and cancel it on a tripwire — so a clean request pays max(guardrail, model) latency, not guardrail + model. The lab runs the guardrail first (sequential) for simpler mechanics; it gets the same safety guarantee (no output escapes a tripwire) at the cost of adding the guardrail's latency to the happy path. Knowing that the real design is concurrent-with-cancellation, and why (hide the guardrail latency on clean traffic), is the distinction between having used guardrails and having reasoned about them.

Sessions: the tenant boundary and the context bomb

Session looks like a convenience — automatic memory, four methods, done. Architecturally it is two things that will hurt you if you treat it as one.

First, session_id is a tenant boundary, not a chat key. The Runner reads every row for a session_id and prepends it to the model's input. If your session_id is scoped to a conversation but not to a user, a collision or a guessable id surfaces one customer's history to another — a cross-tenant data leak dressed up as "memory." The isolation has to live in how you derive and authorize the key (tie it to the authenticated principal), because the store itself will happily return whatever rows match. This is the multi-tenancy blast radius: the failure is silent, it is a privacy incident, and it looks exactly like the feature working.

Second, sessions are where context growth detonates. Every turn re-sends the accumulated history, so token cost per turn grows with conversation length — roughly linear per turn, quadratic over the conversation if you never prune. A naive get_items that returns everything means turn 50 pays for 49 turns of history on every call. Production sessions need pruning or summarization inside get_items (a sliding window, a running summary, semantic compaction). The four-method interface is exactly the seam where you insert that policy — which is why the interface is get/add/pop/clear and not just "a list."

Failure modes and blast radius

Think in terms of what a single bad run can take down:

• Crash mid-run — without Temporal, lose the run and any non-idempotent side effects it half-completed. Blast radius: one request, plus whatever a re-run double-fires. Mitigation: durable wrap, idempotency keys on money-moving tools.
• PII in the final output — the model produces a compliant-looking answer that leaks a card number. The output guardrail is the last gate before a human sees it; if it is missing or on a model too weak to catch it, the incident ships. Blast radius: reputational/regulatory, not just one user.
• Runaway loop — a confused agent burns max_turns of expensive calls before failing. Blast radius: cost, bounded by the budget. This is why the budget is a hard cap and not advisory.
• Guardrail false-positive storm — an over-eager tripwire blocks legitimate traffic. Blast radius: availability. Guardrails are a control plane; a bad guardrail is an outage.

Observability is not optional here

A loop over a stochastic oracle is opaque by construction; you cannot reason about an incident you cannot see. The SDK's built-in tracing — a span per run, turn, model call, tool call, handoff, and guardrail — is the seam that answers the 2 a.m. questions: why did it call that tool, where did it hand off, which guardrail tripped, where did the tokens go. Architecturally, tracing is the reason the RunResult.new_items list exists in the first place: an ordered, inspectable record of every step. In the labs that list is the trace; in production you export spans (with timing and token counts) to a backend. A platform team that adopts the SDK and does not wire the exporter to their observability stack has shipped a black box and will regret it on the first bad night.

The decisions that look wrong but aren't

• State on the stack, not in a store. Looks fragile; is the right default that keeps the 95% case zero-overhead and pushes durability to a pluggable layer for the 5% that needs it.
• Handoff is a tool, so control never returns. Looks like a missing feature ("where's my return value?"); is the decentralized-control design. If you want centralized control, that is a different primitive (agents-as-tools), and the SDK gives you both rather than pretending one covers both.
• Guardrails on a different, cheaper model. Looks like you're cutting corners on safety; is the only configuration where the cost math works, and accuracy is recovered by scoping the guardrail to a narrow yes/no it can nail.

The through-line: the SDK optimizes for the common case being trivial and the hard cases being composable rather than built-in. Your job on the platform side is to know which hard case you're in — durability, cost, isolation, observability — and bring the layer the SDK deliberately left out.