« Phase 04 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 04 Warmup — Context Engineering: Assembly, Memory & Intent Routing

Who this is for: someone who has written prompts and maybe called a RAG pipeline, but has never built the thing that decides what goes into the prompt. By the end you will be able to assemble a budgeted prompt, order it so the model can use it, route an intent (and know when to refuse), and run a three-tier memory — the mechanisms behind "context engineering," the phrase that replaced "prompt engineering" on senior JDs. Nothing here needs a GPU, an API key, or a framework: it is a budget, a priority queue, a classifier, and a memory hierarchy.

Table of Contents

  1. From prompt engineering to context engineering
  2. The context window is a finite, billed budget
  3. Counting the budget: tokens
  4. The anatomy of a prompt, and why order matters
  5. Assembly as a budgeted knapsack: pin, prioritize, truncate, drop
  6. Lost in the middle: position is a control
  7. Context rot: why more context can be worse
  8. Intent classification and routing: why route at all
  9. Workflow collisions: the discipline of refusing to route
  10. Scoring intents: bag-of-words, cosine, threshold, margin
  11. Memory is a hierarchy: working, session, long-term
  12. Compaction: rolling summaries when the buffer evicts
  13. Semantic memory: the hashing trick and recall by similarity
  14. RAG vs long context, and context caching
  15. Common misconceptions
  16. Lab walkthrough
  17. Success criteria
  18. Interview Q&A
  19. References

1. From prompt engineering to context engineering

For two years the craft was prompt engineering: find the magic wording that makes the model behave. That skill has not vanished, but it has been subsumed. Modern agents don't run one hand-written string; they run a payload assembled at runtime from a dozen sources — a system prompt, the user's task, conversation history, a rolling summary of older turns, facts recalled from long-term memory, documents pulled by a retriever, results returned by tools, and few-shot examples. The senior skill is engineering that whole payload: deciding what to include, what to leave out, how to compress it, and how to order it, under a fixed budget. That is context engineering.

The phrase is now literally on the job descriptions (Citi's Lead Agentic AI Engineer lists "context engineering" and "prompt/context management"; Anthropic and others use it in their Claude-Code roles). The shift matters because it changes what you optimize. Prompt engineering optimizes words; context engineering optimizes an information-selection problem under a resource constraint — which is an engineering problem with a budget, a priority order, a cache, and a classifier. As Andrej Karpathy put it, it is "the delicate art and science of filling the context window with just the right information for the next step." The rest of this warmup is the mechanics of doing that.

Why start here? Because juniors reach for a better prompt when the fix is a better context: the model isn't dumb, it just never saw the relevant fact — it was truncated, buried in the middle, or never retrieved. Context engineering is where those bugs live.

2. The context window is a finite, billed budget

Every model has a maximum context length — the number of tokens it can attend to at once (say 8K, 128K, 1M). Two independent forces make that window a budget you spend, not a bucket you fill to the brim:

  • Cost. From Phase 00: you pay per token, output priced 3–5× input, and a ReAct loop resends the whole scratchpad every step, so input tokens grow like \((t+o),n^2/2\) — quadratic in steps. Every token you pack is re-billed on the next turn. "Just put more in the prompt" is a recurring cost decision, not a free one.
  • Latency. Prefill (reading the prompt) scales with input length; a bloated context is a slower first token. The tail (Phase 00's p95) gets worse as the window fills.

And a third, less obvious force — quality — which sections 6 and 7 develop: past some point, adding context lowers accuracy. So the window is a budget bounded three ways — dollars, milliseconds, and correctness — and the assembler's job is to spend it well. Concretely, the invariant your assemble_context must hold is simply: packed tokens ≤ budget, and the must-haves are always present.

3. Counting the budget: tokens

To spend a budget you must measure it, and the unit is the token — a sub-word chunk produced by the model's tokenizer (BPE for GPT/Claude, SentencePiece for many others). There is no clean formula: "hello" is one token, "antidisestablishmentarianism" is several, a Chinese character may be one or more. In production you call the model's own tokenizer (tiktoken, the HF tokenizer) to count exactly, because your budget check must match what the provider bills.

For a deterministic, offline lab we use the famous heuristic: ~4 characters per token for English (equivalently ~0.75 words/token). The lab's count_tokens is:

$$\text{tokens}(s) = \left\lceil \frac{\operatorname{len}(s)}{4} \right\rceil.$$

The one property that must hold for a budget primitive is monotonicity: more characters can never yield fewer tokens. The ceiling of length/4 guarantees it, empty text is 0 tokens, and a test asserts the monotonicity so you never ship a token counter that under-counts a growing string. The estimate is crude — but every decision built on it (pin, drop, truncate) is identical whether the counter is a heuristic or a real BPE tokenizer, which is the point of the miniature. Swapping in tiktoken changes one function and nothing else.

4. The anatomy of a prompt, and why order matters

A well-engineered agent prompt is not a blob; it is a structured document with a conventional order. A typical anatomy, top to bottom:

  1. System / role — who the model is, the rules, the tool contract. (Often cacheable — see §14.)
  2. Long-term memory / persona facts — durable facts about the user or task.
  3. Retrieved documents — the RAG payload for this query.
  4. Rolling summary — a compressed digest of older conversation.
  5. Recent turns — the last few messages verbatim.
  6. The current task / user message — what to do right now.

Two things about this list are load-bearing. First, each item is a section competing for budget, which is why the lab models context as a list of ContextSection(name, text, priority, pinned, truncatable) rather than a string. Second, the order is not arbitrary: the system prompt goes first and the live task goes last precisely because those are the two positions the model attends to most strongly — the subject of §6. Context engineering is choosing the contents (assembly) and the arrangement (ordering) of this document, every turn.

5. Assembly as a budgeted knapsack: pin, prioritize, truncate, drop

Now the core algorithm. You have more candidate sections than fit; you must choose. This is a constrained packing problem — a knapsack with one twist: some items are mandatory. The policy the lab's assemble_context implements, in order:

  1. Pin the must-haves. The system prompt, the task, safety rules — sections marked pinned — are always included. If the pinned set alone exceeds the budget, there is no valid prompt: raise. Silently dropping a pin to fit a search result is a correctness/safety bug (you'd delete the safety rules to quote a web page).
  2. Fill by priority. Add the remaining sections in descending priority. A stable sort keeps equal-priority ties in the caller's order. For each candidate:
    • if it fits, take it;
    • else if it is truncatable, shorten it to the space left and take the head;
    • else drop it.
  3. Report the accounting. Return which sections were included, dropped, and truncated, plus the final token_count. Observability is not optional: when an agent "forgets" something, the first question is was it dropped or truncated?, and the answer must be in the trace.

Truncation on a word boundary. Cutting a section mid-word ("...the transacti") produces garbage sub-word tokens and confuses the model. So truncate_to_tokens adds whole words until the next word plus an elision marker ( […]) would exceed the budget, then stops. The marker matters: it tells the model (and you) that content was removed, so the model doesn't treat a truncated list as complete. The guarantee is count_tokens(result) ≤ max_tokens.

The invariant to hold in your head — and in a test — is: after assembly, tokens ≤ budget and every pinned section survived. Everything else (which optional section got dropped) is policy you can tune.

6. Lost in the middle: position is a control

Here is the finding that turns ordering from cosmetics into engineering. Liu et al. (2023), "Lost in the Middle: How Language Models Use Long Contexts," placed a single relevant document at different positions inside a long context and measured retrieval accuracy. The result was a U-shaped curve: accuracy is highest when the relevant information is at the beginning or the end of the context, and drops — sometimes dramatically — when it sits in the middle. A model can have a fact in its context and still fail to use it because it was buried.

The mechanism is a mix of how attention distributes over long sequences and how models were trained (instructions at the start, the question at the end). You do not need the internals to act on it; you need the design rule:

Put the most important context at the edges. Bury the least important in the middle.

That is exactly what order_for_recall does: sort sections by priority, then deal them alternately to the front and back of the output so the two highest-priority sections land at the two ends and the lowest sits dead center. In build_agent_context this is why the system prompt (priority 100) comes out first and the task (priority 95) comes out last — the two things the model most needs to obey and answer, at the two positions it reads best. This is also the deeper reason retrieval and reranking (Phase 05) matter: it is better to put the five most relevant chunks at the edges than to dump fifty into the middle.

7. Context rot: why more context can be worse

The naive mental model is "the model has a 1M window, so stuff everything in and let it sort it out." This fails, and the failure has a name practitioners now use: context rot. As the context grows, model performance on the actual task tends to degrade, for several compounding reasons:

  • Distraction / dilution. More irrelevant text means more for attention to spread over; the signal-to-noise ratio of the prompt falls, and the model latches onto a plausible-but- wrong nearby passage.
  • Lost in the middle (§6) — the relevant fact is likelier to land in the weak middle zone.
  • Contradiction and staleness. A long history accumulates outdated instructions, tool errors, and abandoned plans that the model may still "obey."
  • Cost and latency (§2) rise the whole time.

The engineering response is curation, not accumulation: retrieve the few most relevant chunks rather than the whole corpus; compact old turns into a summary rather than keep them verbatim (§12); drop or truncate low-value sections; and re-rank so quality, not recency or volume, decides what survives. "A bigger window" is a capability, not a strategy. The strategy is spending the window on the right tokens — which is this entire phase.

8. Intent classification and routing: why route at all

So far we've packed one prompt. But a real assistant serves many kinds of request — billing, cancellation, tech support, small talk — and treating them identically is wasteful and error-prone. Intent routing classifies the incoming query and dispatches it to the right handler before the expensive reasoning happens. Why bother:

  • Cost. Each intent can map to a cheaper, specialized prompt or a smaller model. A FAQ lookup does not need your flagship model; a code-fix does. Routing is one of the biggest cost levers in an agent system (revisited as model cascades in Phase 14).
  • Quality. A prompt focused on one intent (with the right tools and few-shots for that job) beats a bloated do-everything prompt — and keeps §7's rot at bay by only loading the context that intent needs.
  • Safety and correctness. Routing lets you attach the right guardrails and the right data scope per intent (least privilege — Phases 09/10/13).

A router is itself a small classifier. It can be an LLM call ("which of these buckets?"), a trained model, or — as in this lab — a lightweight bag-of-words similarity match that needs no model at all. The important part is not the classifier's sophistication; it is what it does when it is unsure, which is the next section.

9. Workflow collisions: the discipline of refusing to route

Citi's JD asks, in so many words, for "prompt and intent-classification logic to avoid workflow collisions." A workflow collision is what happens when a request matches two intents at once and the router picks one anyway. "Cancel my subscription and refund the last charge" is both a cancellation and a billing request; route it to the cancel pipeline and the refund never happens (and vice versa). The user gets half-served and the failure is silent.

The fix is not a better classifier — it is a router that knows when it doesn't know and refuses to route, handing off to a clarify turn or a human instead of guessing. There are two distinct "don't know" conditions, and a good router distinguishes them:

  • Low confidence. The top intent's score is below a threshold — the query matches nothing well ("what's the weather?" to a billing bot). Don't force it into the nearest bucket; fall back.
  • Ambiguous tie (the collision). The top two intents score within a small margin of each other — the query matches two things equally. Routing to either is a coin flip that mishandles the other half. Fall back and clarify.

The lab's route returns a RouteDecision with a fallback flag and a reason (low_confidence vs ambiguous_tie) for exactly this. The senior insight mirrors Phase 00's "the most senior thing is to say this shouldn't be an agent": the router's most valuable output is often the refusal. A confident wrong route is worse than an honest "which did you mean?"

10. Scoring intents: bag-of-words, cosine, threshold, margin

How does the lab score an intent without a model? Each intent is registered with a set of keywords — a bag-of-words prototype. A bag-of-words vector is just a token→count map: order and grammar are thrown away, only which words appear (and how often) is kept. To compare the query's bag \(q\) to an intent's bag \(d\), use cosine similarity — the cosine of the angle between the two vectors, which measures direction (shared vocabulary) independent of magnitude (length):

$$\cos(q,d) = \frac{\sum_i q_i, d_i}{\sqrt{\sum_i q_i^2}\ \sqrt{\sum_i d_i^2}} \in [0,1].$$

1.0 means identical direction (same words in the same proportions); 0.0 means no shared tokens. The router computes \(\cos(q,d)\) for every intent, sorts, and applies the two-gate decision from §9:

  • if top_score < threshold → fallback (low_confidence);
  • else if top_score - second_score < margin (and the second actually overlaps) → fallback (ambiguous_tie);
  • else → route to the top intent.

Worked example from the lab: intent billing = {refund, charge, invoice, payment, money}, intent cancel = {cancel, charge, subscription, stop, account}. The query "cancel and refund charge" shares {refund, charge} with billing and {cancel, charge} with cancel — two tokens each, equal cosine — so top - second = 0 < margin: an ambiguous tie, and the router refuses. Change the query to "I need a refund for my invoice payment" and billing wins cleanly (three shared tokens vs zero) — a confident route. Same code, and the threshold and margin are the two dials you tune against a labeled set. A production semantic router (e.g. the semantic-router library) swaps the bag-of-words for real embeddings, but the threshold/margin/fallback logic is identical.

11. Memory is a hierarchy: working, session, long-term

An LLM is stateless: it remembers nothing between calls except what you put back in the prompt. So "agent memory" is your data structure, not the model's. And it must be a hierarchy, because the three things you need — recency, continuity, and durable recall — have different budgets and access patterns. The canonical three tiers (a deliberate echo of a CPU's registers → RAM → disk):

TierWhat it holdsBounded byIn this lab
Workingthe last few turns, verbatima small token/turn budgeta FIFO deque(maxlen)
Sessiona running summary of older turnsone compact stringthe rolling summary
Long-termdurable facts, recalled on demanda vector store's sizethe semantic store

This is not academic — it is exactly what shipping products implement. Claude Code keeps your live turns, a CLAUDE.md of durable project facts, and compacts the conversation when it grows. Cursor has "memories." ChatGPT memory persists facts across sessions and injects the relevant ones. MemGPT/Letta and Mem0 formalize the tiers with paging between them. The lab's TieredMemory is the minimal honest version: a buffer that evicts, a summary that grows, and a semantic store you query.

The three tiers feed the assembler (§5): memory.context(query) returns the recent turns (highest priority — the live conversation), the recalled facts, and the summary, each as a ContextSection, and assemble_context fits them into the turn's budget alongside the system prompt and retrieved docs.

12. Compaction: rolling summaries when the buffer evicts

The working buffer is bounded, so old turns fall out — but you can't just delete them or the agent forgets the first half of the conversation. Compaction (a.k.a. conversation summarization) is the answer: when a turn is evicted from the buffer, fold it into a rolling summary instead of dropping it. You trade fidelity for a bounded token footprint — the summary is lossy, but it is small and it keeps the thread alive across a 40-turn chat that would never fit verbatim.

In the lab, TieredMemory.add_turn evicts the oldest turn when the buffer overflows and recomputes summary = summarizer(evicted_turns), where summarizer is an injected Callable[[list[Turn]], str]. This injection is the same seam from the Lab Standard: in production the summarizer is an LLM call ("summarize these turns in two sentences"); in the lab it is a pure function so eviction→summary is deterministic and testable. A test adds buffer_size + 1 turns and asserts the first turn's content now appears in the summary — proving nothing was silently lost.

Real systems get fancier — summarize in chunks, keep entities/decisions structured, re- summarize the summary when it grows — but the mechanism is this: eviction triggers compaction, and compaction is a summarizer you inject. This is Claude Code's /compact and the "auto-compaction" every long-running agent needs so the window is spent on the recent and the relevant, not the stale and verbatim.

13. Semantic memory: the hashing trick and recall by similarity

The long-term tier stores facts you may need turns or sessions later, and you can't keep them all in the prompt. So you store them out-of-context and recall only the ones relevant to the current query — the same retrieve-by-similarity idea as RAG (Phase 05), applied to memory. Each fact is stored with an embedding (a dense vector), and recall returns the top-\(k\) facts whose embeddings are most cosine-similar to the query's embedding.

To keep the lab offline and deterministic we build the embedder from stdlib with the hashing trick (feature hashing). For each token: hash it with hashlib.sha256 — not Python's builtin hash(), which is salted per process and would make recall differ across runs — map the digest to a bucket in [0, dim) and a sign, and accumulate the signed count:

for token in tokenize(text):
    d      = sha256(token)
    bucket = int(d[:8]) % dim
    sign   = +1 if d[8] is odd else -1
    vec[bucket] += sign

Two texts that share tokens land signed mass in the same buckets, so their vectors correlate and their cosine is high; texts with no shared vocabulary get near-orthogonal vectors and low cosine. That is enough to make recall work and be testable: the lab stores three facts, queries "python decorators add behavior to functions," and recall returns the python decorators fact first because it shares the most salient tokens. The crucial limitation, which you should say out loud in an interview: this captures lexical overlap, not meaning — "car" and "automobile" look unrelated. A real embedding model fixes that; the mechanism (text → vector, compare by cosine, take top-k) is identical, which is why Phase 05 can swap the embedder without touching the recall logic.

14. RAG vs long context, and context caching

Two questions always follow from "the window is a budget," and both are forward references you should be able to frame now.

RAG vs long context. If a model has a 1M-token window, why retrieve at all — why not paste the whole knowledge base in? Because of everything in §2 and §7: pasting a corpus is expensive (re-billed every turn), slow (prefill), and less accurate (context rot, lost-in-the-middle). Retrieval — fetch the few most relevant chunks and put them at the edges — is usually cheaper and better than a giant context, which is why RAG did not die when windows grew. The honest nuance: for a small, stable corpus that fits comfortably, long-context can win on simplicity; for a large or changing one, retrieval wins. That build vs paste tradeoff is the whole of Phase 05/06; this phase gives you the budget lens to reason about it.

Context caching. The system prompt, tool schemas, and few-shot examples are often identical across turns. Re-sending and re-processing them every call is wasted money and latency. Prompt/context caching (Anthropic's prompt caching, provider-side KV caching) stores the processed prefix so repeated tokens are billed at a steep discount and skip prefill. This is why the prompt anatomy in §4 puts stable content first (cacheable prefix) and variable content (the task) last: you assemble for the cache boundary as well as for recall. The mechanics — cache keys, hit rates, prefix vs semantic caching — are Phase 14; the design habit starts here.

15. Common misconceptions

  • "Bigger context window means I don't have to curate." No — a bigger window is a bigger budget, and §7 says spending it all lowers quality. Curate regardless of window size.
  • "Order doesn't matter, the model reads everything." §6: recall is U-shaped; a fact in the middle can be effectively invisible. Order is a control.
  • "More retrieved chunks is safer." Diminishing then negative returns: the relevant one gets diluted and buried. Retrieve fewer, rank better, place at the edges.
  • "Truncate anywhere to make it fit." Mid-word cuts create garbage tokens and drop the signal that content was removed. Cut on word boundaries and leave an elision marker.
  • "A router should always pick the best intent." The best intent might be a coin flip. On low confidence or a tie, refusing to route (clarify) beats a confident wrong route — that is the "workflow collision" discipline.
  • "Memory is just the chat history." History is one tier. Without compaction it overflows; without a semantic store it forgets anything older than the buffer. Memory is a hierarchy.
  • "Summarization is lossless enough." Compaction is lossy by design — it trades fidelity for a bounded footprint. Keep the recent turns verbatim; summarize only what fell out.

16. Lab walkthrough

Open lab-01-context-assembler-router/ and fill the TODOs top to bottom — the file is ordered to match this warmup:

  1. Tokens — count_tokens (ceil(len/4), monotonic) and truncate_to_tokens (add whole words until the next word + ELISION would overflow; guarantee ≤ max_tokens).
  2. Assembly — assemble_context (pin → fill by descending priority → truncate-or-drop → pack in given order; raise on pinned overflow; token_count ≤ budget) and order_for_recall (sort by priority, deal front/back so the top two are at the edges).
  3. Similarity — bag_of_words, cosine_sparse, cosine_dense (guard zero norms).
  4. Routing — IntentRouter.register (keywords → bag) and route (score, then the four-way no_intents / low_confidence / ambiguous_tie / match decision).
  5. Memory — embed (the hashlib hashing trick), TieredMemory.add_turn (evict → summarizer(evicted)), remember, recall (cosine top-k), context / MemoryContext.as_sections.
  6. Integration — build_agent_context (route → gather → order_for_recall → assemble_context).

Run LAB_MODULE=solution pytest test_lab.py -v first to see green, then make your lab.py match. Finish by reading solution.py's main() output — the router picking an intent and refusing a collision, the assembler dropping and truncating to fit while keeping the pins, and memory summarizing an evicted turn and recalling a relevant fact.

17. Success criteria

  • You can explain, from the U-shaped curve, why order_for_recall brackets the two highest-priority sections at the two ends — and why that beats dumping everything in.
  • Your assemble_context holds the invariant (tokens ≤ budget, pins always present), drops lowest-priority first, truncates on a word boundary, and raises on pinned overflow.
  • You can state the two conditions under which a router should refuse to route and name the "workflow collision" the ambiguous-tie branch prevents.
  • You can name the three memory tiers, say what compaction trades, and map each tier to Claude Code / Cursor / ChatGPT.
  • You can argue RAG-vs-long-context and context caching as budget decisions.
  • All 30 lab tests pass under both lab and solution.

18. Interview Q&A

Q: What is "context engineering," and how is it different from prompt engineering? A: Prompt engineering is optimizing one hand-written string; context engineering is engineering the whole runtime payload — system prompt, memory, retrieved docs, tool results, few-shots — into a finite token budget, in an order the model can use, having routed the request first. It's an information-selection problem under a resource constraint: a budget, a priority queue, a cache, and a classifier, not wordsmithing. It's the phrase that's now on senior JDs because that's what the job actually is.

Q: Your agent has the right document in context but still answers wrong. What do you check? A: Position. "Lost in the middle" (Liu et al. 2023) says recall is U-shaped — strong at the edges, weak in the middle — so a fact buried in a long context can be effectively invisible. I'd move the relevant chunks to the beginning/end (reorder), and more fundamentally retrieve fewer, better-ranked chunks so the signal isn't diluted (context rot). Bigger window isn't the fix; curation and ordering are.

Q: You have a 1M-token model. Do you still need RAG? A: Usually yes. Pasting a corpus is expensive (re-billed every turn), slow (prefill), and often less accurate than retrieval because of context rot and lost-in-the-middle. Retrieval fetches the few most relevant chunks and lets you place them well. For a small, stable corpus that fits comfortably, long-context can win on simplicity; for a large or changing one, retrieval wins on cost and quality. It's a budget decision, not a capability one.

Q: Design intent routing that avoids "workflow collisions." A: Score the query against each intent (embeddings/bag-of-words cosine, or an LLM classifier). Then two gates before you dispatch: if the top score is below a confidence threshold, the query matches nothing — fall back to clarify; if the top two scores are within a margin, it's an ambiguous tie (e.g. "cancel and refund") — also fall back, because routing to either mishandles the other half. The router's most valuable output is the refusal. I'd tune threshold/margin against a labeled set and log route-vs-fallback to watch the false-route rate.

Q: How do you keep a 100-turn conversation inside an 8K window? A: A memory hierarchy. Keep the last few turns verbatim in a working buffer; when turns evict, compact them into a rolling summary (an LLM summarizer) so the thread survives lossily but cheaply; and put durable facts in a semantic long-term store you recall from by similarity per query. Assemble summary + recalled facts + recent turns + system prompt into the budget each turn, pinning the system prompt and ordering for recall. That's what Claude Code's compaction and ChatGPT memory do.

Q: Why is truncation done on a word boundary with a marker? A: Cutting mid-word produces garbage sub-word tokens that confuse the model, and cutting without a marker makes a truncated list look complete, so the model reasons over partial data as if it were whole. Adding whole words up to the budget and appending an elision marker keeps the tokens clean and signals that content was removed — both matter for correctness.

Q: You inject the summarizer and the embedder as callables. Why? A: Determinism and testability — the same seam the whole track uses for the LLM. The real summarizer is a non-deterministic model call and the real embedder is a learned model; injecting them as pure functions makes eviction→summary and recall reproducible so a unit test can assert the exact behavior, and swapping in the real thing later touches one seam, not the assembler/router/ memory logic.

19. References

  • Liu, Lin, Hewitt, et al., Lost in the Middle: How Language Models Use Long Contexts (2023) — the U-shaped position curve. https://arxiv.org/abs/2307.03172
  • Karpathy, on "context engineering" (2025) — "filling the context window with just the right information for the next step." https://x.com/karpathy/status/1937902205765607626
  • Anthropic, Effective context engineering for AI agents (2025). https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
  • Anthropic, Prompt caching docs — cacheable stable prefixes. https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
  • Anthropic, Building Effective Agents (2024) — routing as a workflow pattern. https://www.anthropic.com/research/building-effective-agents
  • Packer et al., MemGPT: Towards LLMs as Operating Systems (2023) — paged memory tiers. https://arxiv.org/abs/2310.08560
  • semantic-router (Aurelio Labs) — production intent routing over embeddings. https://github.com/aurelio-labs/semantic-router
  • Weinberger et al., Feature Hashing for Large Scale Multitask Learning (2009) — the hashing trick. https://arxiv.org/abs/0902.2206
  • Anthropic, Manage context on Claude Code — compaction / CLAUDE.md. https://docs.anthropic.com/en/docs/claude-code/costs