« Phase 04 README · Warmup · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 04 — Hitchhiker's Guide

The compressed practitioner tour. Read the WARMUP for the derivations; this is the stuff you say in the meeting.

30-second mental model

Context engineering is prompt engineering grown up: you don't write one string, you assemble the whole payload — system prompt, memory, retrieved docs, tool results — into a finite token budget, in an order the model can use, having routed the request first. The window is a budget you pack (pin the must-haves, add by priority, truncate or drop the rest), position is a control (lost-in-the-middle → important stuff at the edges), the router's best output is often a refusal (don't route an ambiguous "cancel and refund"), and memory is a hierarchy (buffer → rolling summary → semantic recall), not a log.

The numbers to tattoo on your arm

Framework one-liners

• LangChain / LlamaIndex = retrievers + "context/prompt compressors" + memory classes; the assembler and memory tiers you built, with more knobs.
• semantic-router = intent routing over embeddings; your IntentRouter with real vectors and the same threshold/fallback logic.
• Claude Code = live turns + CLAUDE.md (long-term) + auto-compaction; the three tiers, shipped.
• ChatGPT memory / Cursor memories = persistent long-term facts injected by relevance.
• MemGPT / Letta, Mem0 = memory as an OS: paging between working/session/long-term tiers.
• Anthropic prompt caching = cache the stable prefix so it's cheap and skips prefill.

War stories

• The 1M-window agent that got dumber. Team pasted the whole handbook into context "because it fits." Accuracy dropped and latency tripled — context rot plus lost-in-the-middle. Switching to retrieve-top-5-and-put-them-at-the-edges beat the giant context on both quality and cost.
• The refund that never happened. "Cancel my plan and refund the last charge" routed to the cancel workflow; the refund silently vanished. A workflow collision — the router should have detected the tie and asked which one first. Added an ambiguous-tie fallback and the class of bug disappeared.
• The agent with amnesia at turn 12. A fixed 10-turn buffer, no compaction — everything older just fell off a cliff. Added a rolling summary on eviction and a semantic store for durable facts; the 40-turn conversations held together.
• The truncation that lied. A list of open tickets got cut mid-item with no marker; the model confidently reported "3 open tickets" from a truncated 30. Word-boundary cut + elision marker fixed it.

Vocabulary

Context engineering (assemble the whole payload under budget) · Token (sub-word budget unit, ~4 chars) · Context window (max tokens, a budget) · Pinned section (must-include) · Truncatable (may shorten to fit) · Lost in the middle (U-shaped recall) · Context rot (quality falls as context grows) · Intent routing (classify → dispatch) · Fallback / clarify (refuse to route) · Workflow collision (ambiguous multi-intent request) · Working / session / long-term memory · Compaction (summarize evicted turns) · Semantic recall (top-k by embedding cosine) · Hashing trick (stdlib embedder) · Context caching (cheap reused prefix).

Beginner mistakes

1. Reaching for a bigger window instead of curating the context (context rot).
2. Ignoring order — burying the key fact in the middle where the model won't read it.
3. Truncating mid-word with no elision marker (garbage tokens, silent data loss).
4. Building a router that always picks a winner instead of clarifying on a tie.
5. Treating chat history as "memory" — no compaction (overflow) and no semantic store (amnesia).
6. Counting tokens with a guess that isn't monotonic, so the budget check under-counts.
7. Putting variable content before stable content and killing your prompt-cache hit rate.