« Phase 20 README · Warmup · Hitchhiker's · Deep Dive · Core Contributor · Staff Notes

Phase 20 — Principal Deep Dive: Amazon Bedrock AgentCore

Zoom out from the mechanism to the platform. AgentCore's bet is that the seven hard parts of running an agent — hosting, isolation, tool access, authorization, identity, memory, observability — are horizontal infrastructure that look the same whether the loop is a LangGraph graph or a CrewAI crew, so AWS provides them as composable services and you spend your budget on agent logic. This doc is about the consequences of that bet: the scaling envelope, the cost and latency math, the blast radius, and the decisions that look wrong until you see the constraint they solve.

The architecture is a data-plane / control-plane split with isolation as the substrate

Read the primitives as planes. Runtime is the data plane — where the agent loop actually executes, one microVM per session. Gateway + Policy + Identity are the tool control plane — every side-effecting action funnels through a single authenticate → authorize → execute chokepoint. Memory is the state plane — durable, namespaced, decoupled from any session's lifetime. Observability is the cross-cut that makes all three legible. The substrate under everything is Firecracker-class microVM isolation: it is not a feature bolted onto Runtime, it is the property that lets a single multi-tenant service host thousands of untrusted agent sessions without a shared-kernel compromise turning one tenant's prompt injection into every tenant's breach.

That framing tells you where the bodies are buried: the Gateway is a serialization point. Every tool call from every agent passes Identity and Policy before it executes. That is the correct design — a deterministic authorization chokepoint is exactly what you want on the trusted side of the boundary — but it means Policy latency and Gateway availability are on the hot path of every action the fleet takes. Budget for it; cache token validation; keep policy sets small (evaluation is O(rules) per call, per the Deep Dive's two-pass scan).

Cold-start math and warm-pool sizing

The microVM model trades a per-session boot cost for hard isolation. Firecracker boots in roughly the low-hundreds of milliseconds (the technology under Lambda and Fargate), plus your container and agent-init time on top. So a cold session pays boot + init once; a warm session (reused session_id) pays neither and can also see in-session scratchpad state. The lab models exactly this: cold_start=True on first invoke, state surviving in session.store across turns, is_warm flipping only after a completed invocation.

The capacity question a principal owns: how big is the warm pool, and what is the cold-start tail? Sketch the math. If new-session arrivals are λ per second and each cold start adds c seconds of latency, then a fraction of requests equal to (new sessions)/(total requests) pays c. If sessions average k turns, only 1/k of invocations are cold, so cold start contributes to the p-tail roughly in proportion to 1/k. A chat agent with k≈8 turns hides cold start from ~88% of invocations; a fire-and-forget agent with k=1 pays it every time and cold start dominates your p95. That single ratio — turns per session — decides whether the microVM model is cheap or ruinous for your workload, and it is the first number to establish before adopting Runtime.

Warm-pool sizing follows the same logic as any connection pool. The lab's max_sessions LRU eviction is the miniature: too small a pool and you evict sessions that are about to be reused, converting warm hits into cold starts (a thrash regime); too large and you pay to hold idle microVMs. The real Runtime scales to zero and reclaims idle sessions on a TTL, so the knob you actually own is session lifetime — how long to keep a conversation warm against the cost of holding its microVM. That is a latency-versus-cost decision, not a correctness one.

Where state lives is a latency, cost, and blast-radius decision

There are three places state can live, and choosing among them is a principal-level call:

• Warm-session store (short-term, in the microVM): fastest, free to read, but volatile and lost on eviction/TTL. Good for the current conversation's scratchpad.
• Long-term Memory (namespaced records): survives sessions and is shareable across agents, but every read is a retrieval (embedding + ANN in production) with its own latency and cost, and every write is a consolidation.
• Neither (recompute): cheapest to store, most expensive to reproduce.

Put a customer's preferences in warm-session state and they vanish between sessions — the "amnesiac agent." Put the entire chat transcript in long-term Memory verbatim and you pay to embed and store noise, and retrieval quality drops as the namespace fills with low-signal records. The design intent behind strategies is precisely this: consolidate turns into a few durable, deduplicated records (a summary, extracted facts, learned preferences) so long-term memory stays small and high-signal. Idempotent consolidation (dedup on content) is what keeps the namespace from growing linearly with turns.

Multi-tenancy and blast radius

The blast-radius analysis is the reason enterprises adopt this. In a shared-process agent server, one compromised session — a prompt injection that writes a hostile file, an escaped code execution — can read every other in-flight session's memory, because they share an address space and a kernel. The blast radius is the whole process, i.e. every tenant currently served. Under per-session microVMs, a compromise is contained to one session's microVM: separate kernel, separate memory, separate filesystem, torn down at session end. The blast radius is one session. That containment is not achievable by careful coding; it requires the hardware-virtualization boundary. This is the single most important sentence to be able to defend in an architecture review.

Memory multi-tenancy has its own blast-radius knob: the namespace template. Keying records by {actor} (/facts/user-42) scopes memory to a user and shares it across that user's agents and sessions — good. Get the template wrong — key by agent instead of actor, or forget to template at all so every tenant writes the same namespace — and you have built a cross-tenant memory leak in the state plane, the exact failure the Runtime prevents in the data plane. Namespace design is a security decision wearing a naming-convention costume.

Failure modes to design against

• Cold-start storm. A fleet-wide event (deploy, cache flush, traffic spike of net-new sessions) makes most invocations cold simultaneously, spiking p99. Mitigate with pre-warming and by not minting a new session_id per request.
• Policy misconfiguration. Default-deny is your friend here: a forgotten permit fails closed (tool denied), not open. The dangerous failure is an over-broad permit or a forbid with a buggy when predicate. Test the denials, not just the allows — the empty-audit-log assertion is the pattern.
• Gateway/Identity dependency. Because they gate every action, an outage or latency spike there degrades the whole fleet's ability to act. Treat them as tier-0 dependencies with the SLOs to match.
• Memory retrieval degradation. As namespaces fill, relevance drops and latency rises; low-signal records crowd out high-signal ones. Consolidation quality and dedup are the levers.
• Long-running-invocation timeouts. The extended/async runtime lets sessions persist for a long duration; a naive client bound to a short request timeout will abandon work the Runtime is still doing. Match client timeouts to the workload.

"Looks wrong but is intentional"

• The entrypoint is a plain (payload, context) function, not a rich agent interface. That minimalism is the framework-agnostic property: anything that fits the signature runs — LangGraph, CrewAI, a raw loop. A richer contract would leak an opinion about the agent's shape and break the "bring your own framework" pitch.
• Policy is deterministic and dumb, not an LLM. A model asked "is this safe?" is stochastic and injectable. Cedar's determinism is the feature: same request, same decision, no argument. Putting the "smart" check here would be a downgrade.
• Memory is keyed by actor, not by agent. Looks like it couples agents; actually it is what makes memory shareable — the booking agent's learning helps the concierge agent — while namespaces keep tenants apart.
• Services are à la carte, not a bundle. Looks like more integration work than a managed agent; actually it is what lets you adopt Gateway today and Memory next quarter without betting the whole system on one product — the deliberate inversion of the older Bedrock Agents monolith.

Take the whole thing as one claim: AgentCore does not make the agent smarter; it makes the agent survivable in production — isolated, authorized, remembered, and observed — and it does so with primitives whose costs and failure modes you can now put numbers on. The Core Contributor notes go under the hood of how AWS actually implements them.