Lab 01 — Context Assembler, Intent Router & Tiered Memory

Phase 04 · Lab 01 · Phase README · Warmup

The problem

Phase 00 proved the context window is a finite, billed budget and that a ReAct scratchpad grows quadratically. So on every turn you face the real question of context engineering: given more candidate context than fits, what do you put in the prompt, and in what order? Get it wrong three ways and the agent fails:

1. Overflow — you exceed the window (a hard error) or silently drop the system prompt.
2. Lost in the middle — you pack the right facts but bury them where the model can't see them; recall is U-shaped in position (Liu et al. 2023).
3. Workflow collision — you route a "cancel and refund" request to a single pipeline because your intent classifier guessed instead of asking (Citi's JD calls this out by name).

You will build the three mechanisms that solve these — an assembler, a router, and a tiered memory — as small, exact, tested functions, then compose them into one build_agent_context that produces a complete, budgeted, recall-ordered prompt.

What you build

File map

Run it

pytest test_lab.py -v                       # your lab.py (red until you implement)
LAB_MODULE=solution pytest test_lab.py -v   # the reference (green)
python solution.py                          # worked example

Success criteria

• assemble_context never exceeds the budget, always keeps pinned sections, drops the lowest-priority sections first, and truncates a truncatable block instead of dropping it — and raises when the pinned set alone overflows.
• order_for_recall puts the two highest-priority sections at the two ends and the lowest in the middle, and you can explain why from the lost-in-the-middle curve.
• IntentRouter.route returns a clean intent on a good match, but a fallback on a low-confidence query and on an ambiguous tie — and you can name the "workflow collision" it prevents.
• TieredMemory evicts oldest-first, folds evicted turns into the rolling summary, and recall returns the semantically nearest fact — deterministically.
• All 30 tests pass under both lab and solution.

How this maps to the real stack

• assemble_context is what every agent framework does under the hood when it builds a prompt: LangChain/LlamaIndex "context/prompt compressors," the OpenAI Agents SDK's session context, and Claude Code's context assembly all pick what fits under a token budget. The pin/priority/truncate policy here is the same shape; a real system adds per-source compressors and a real tokenizer.
• order_for_recall is the operational form of Liu et al., "Lost in the Middle" (2023): retrievers and rerankers (Phase 05) exist partly so the few most relevant chunks go at the edges instead of dumping 50 into the middle.
• IntentRouter is a from-scratch semantic router (cf. the semantic-router library, NeMo Guardrails' flows, Rasa's intent classifier). In production the fallback branch gates a clarify turn or a human handoff, and each intent maps to a cheaper, specialized prompt or model — the routing cost lever revisited in Phase 14.
• TieredMemory is the working/session/long-term hierarchy behind Claude Code (CLAUDE.md + live context + compaction), Cursor memories, ChatGPT memory, and frameworks like MemGPT/Letta and Mem0. The rolling summary is compaction; the semantic store is long-term memory backed by a vector DB in the real thing (Phase 05/06).
• embed is the hashing trick (feature hashing). Real systems call a learned embedding model; the mechanism — token → vector, compare by cosine — is identical.

Limits of the miniature. count_tokens is a char estimate, not a real BPE tokenizer; the embedder captures lexical overlap, not meaning (so "car" and "automobile" look unrelated); the router is bag-of-words, not a trained classifier; and the summarizer is an injected pure function, not an LLM. The decisions — budget, pin, drop, truncate, order, route-or-clarify, evict-and-summarize, recall-by-similarity — are exactly the production ones.

Extensions (your own machine)

• Swap count_tokens for a real tokenizer (tiktoken) and embed for a real embedding model; the assembler, router, and memory code should not change — only the two seams.
• Add a per-source compressor: instead of truncating on a word boundary, summarize an over-budget section with an (injected) summarizer before packing.
• Give the router confidence calibration: log routed vs. fallback decisions and tune threshold/margin against a labeled set; measure the false-route rate.
• Add memory decay / TTL and dedup to the semantic store so it doesn't grow without bound, and a pin_fact API for facts that must never be evicted.

Interview / resume signal

"Built a context-engineering core from scratch — a budgeted prompt assembler (pin / prioritize / truncate / drop with a lost-in-the-middle ordering), a bag-of-words intent router that returns a clarify-fallback on low-confidence and ambiguous 'workflow collision' queries instead of guessing, and a tiered memory (FIFO buffer + rolling compaction summary + hashing-embedder semantic recall) — all pure-stdlib and deterministic so every packing and recall decision is unit-tested."