Agent Memory and Context (Project Memory, Rules, and Long-Term Recall)

Reference deep-dive (a conceptual walkthrough — not a 15-section concept doc), companion to What Happens When You Prompt an LLM Agent. Up: References · Builds on: what-happens §0 (context) · §3 (reduction) · §4 (storage)

"Memory" is one of the most overloaded words in the agent world: it names four different things, and tools use it loosely. This document defines each precisely, ties them to the stateless-model truth, and maps how Claude Code, Cursor, GitHub Copilot, and BMAD implement them — so "project memory," "rules," "context," and "long-term memory" stop being interchangeable buzzwords.

Accuracy note. The concepts (statelessness, in-context vs retrieved memory) are universal. Specific product features (Claude Code's CLAUDE.md, Cursor "Memories", Copilot instruction files, BMAD sharding, AGENTS.md) evolve quickly — treat the tool details as the shape of each approach and verify current specifics in each tool's docs (the curriculum's "go to primary sources" rule, Phase 0.02).

Contents

• The one big idea
• 1. The four kinds of memory
• 2. The two delivery strategies: always-loaded vs retrieved
• 3. Project memory, in depth
• 4. Long-term / learned memory, in depth
• 5. Tool by tool
• 6. The agent-memory research taxonomy
• 7. Design guidance: what goes where
• 8. Best practices
• References

The one big idea

Burn in the same law that anchors the what-happens doc:

The model is stateless. It remembers nothing. Every form of "memory" is the harness deciding what text to put back into the limited context window on each call.

So "memory" is never in the model — it's a strategy for choosing what to re-inject into context (what-happens §0). Two questions separate the kinds of memory:

1. Lifespan — does it live for one turn, one conversation, or forever?
2. Delivery — is it always loaded into context, or retrieved on demand when relevant?

Everything below is an answer to those two questions.

1. The four kinds of memory

• Working memory is ephemeral and capped by the window. When it fills, the harness compacts/truncates/retrieves (§3).
• Session memory is just working memory persisted to disk so a conversation survives a restart (--resume). The model still remembers nothing — the transcript is replayed (§8).
• Project memory and long-term memory are the two that people mean by "memory" in the configuration sense — and they differ in one critical way (next section).

2. The two delivery strategies: always-loaded vs retrieved

The split between project memory and long-term memory is really a split between two delivery strategies, and the trade-off is identical to stuffing-vs-retrieval (§3.4):

ALWAYS-LOADED (project memory)              RETRIEVED ON DEMAND (long-term memory)
┌──────────────────────────────┐            ┌───────────────────────────────────────┐
│ small, curated instruction    │            │ a large store of facts (files / vectors)│
│ file in the STABLE PREFIX      │            │ → embed the situation → fetch top-k     │
│ → on EVERY call, cached (§2)   │            │ → inject only those into context        │
└──────────────────────────────┘            └───────────────────────────────────────┘
  "standing orders"                            "a filing cabinet you open when needed"
  cost: token tax every call                   cost: a retrieval step; can MISS
  must stay SMALL + STABLE                      scales to unlimited facts

• Always-loaded is simple and reliable (it's always there) but it taxes every call with tokens and must be small and stable to stay cache-friendly (§4 on efficient static files). Use it for the handful of rules that apply all the time.
• Retrieved scales to unlimited facts and keeps the context small, but it can miss (if retrieval doesn't surface the right fact, the model never sees it — a silent Law-1 failure). Use it for the long tail of facts that are occasionally relevant.

This is the single most useful distinction in the whole topic. "Project memory" = always-loaded curated rules. "Long-term memory" = a retrieved store. They are complementary, not competitors.

3. Project memory, in depth

Project memory is a small, durable, repo-scoped file (or set of files) of standing instructions and facts that the agent loads into context at the start of every session. It answers "what should you always know when working in this codebase?" — conventions, architecture notes, build/test commands, do's and don'ts, the project's vocabulary.

Mechanically it is prepended to the system area of the context (§4.2), so it sits in the stable, cached prefix (§2). Two consequences:

1. Every token in it is paid on every call — keep it tight (it's the efficient-static-file problem). A bloated 2,000-token memory file taxes the whole session.
2. Keep it stable — editing it busts the prompt cache for the rest of the prefix. Don't put timestamps/volatile content in it.

It's the agent equivalent of a README for the AI: onboarding docs that a teammate would read once and remember — except the stateless agent must "re-read" them every call, which is exactly why they get auto-injected.

4. Long-term / learned memory, in depth

Long-term memory is a growing store of facts the agent writes over time and recalls later — across conversations and often across projects. It's how an agent "learns" your preferences, past decisions, and recurring context without you re-stating them.

The loop has two halves:

• Write/extract: after (or during) a session, salient facts are saved — sometimes explicitly ("remember that…"), sometimes auto-extracted by the system. Each fact is stored as text (a file, a row, a vector-store entry), usually with a short description/embedding for later matching.
• Recall/retrieve: on a new task, the system finds the relevant facts (by keyword/semantic similarity) and injects only those into context — the retrieval-not-stuffing pattern, because the whole store is far too big to always load.

A common hybrid (used by Claude Code's file-based memory, and what produced this very document's sibling fact-file) is an always-loaded index + retrieved bodies: a tiny MEMORY.md index of one-line pointers rides in context every session (so the agent knows what it knows), while the full fact-files are read only when a pointer looks relevant. That gets you reliable awareness (always-loaded index) without the token tax of loading every fact (retrieved bodies).

Dedicated memory systems formalize this:

• Letta (formerly MemGPT) — treats the context window like RAM and an external store like disk, with the model self-editing its memory blocks and "paging" facts in/out as needed.
• mem0 — a memory layer that extracts facts from conversations, stores them in a vector DB, and retrieves relevant ones per query.
• LangChain / LlamaIndex memory modules, and bespoke vector-store setups, do the same write→embed→retrieve loop.

What "I'll update project memory" meant earlier. The store I updated is technically long-term/learned memory: a MEMORY.md-indexed set of fact-files under this project's folder. I appended a fact (curriculum phase status) so a future session doesn't re-derive it. The index is always-loaded; the fact-file is recalled when relevant. It's "project"-scoped only because it lives under the project's directory — which is why the terms blur in casual use.

5. Tool by tool: Claude Code, Cursor, Copilot, BMAD

All four obey the same law; they differ in file conventions and how much retrieval/auto-learning they add.

Reading the table: the project memory column is the same idea everywhere — a small always-loaded instruction file scoped to the repo, just with different filenames. The long-term column is where tools diverge: Cursor and Claude Code add auto-learning/recall; Copilot leans on explicit instruction files + per-query workspace retrieval.

BMAD is the clearest teaching example. It has no magic memory — it deliberately writes structured documents (PRD → architecture → sharded stories) because the model is stateless and the window is finite. The docs are the durable memory; sharding is retrieval; the persona agents each get a curated context slice. It's the same physics as CLAUDE.md + retrieval, formalized into an agile workflow.

The convergence to watch: AGENTS.md. An emerging cross-tool standard for project memory (used by OpenAI's Codex CLI, supported by GitHub Copilot's coding agent, and increasingly others) — one instruction file many agents read, so you don't maintain CLAUDE.md + .cursorrules + copilot-instructions.md in parallel.

6. The agent-memory research taxonomy

Academic and framework literature (and tools like Letta/mem0) borrow human-memory terms. They all map back to "stored text the harness retrieves into context":

None of these are special model abilities — each is text written to a store and retrieved into context when relevant. The taxonomy is a useful organizing vocabulary, not a different mechanism.

7. Design guidance: what goes where

Use the always-loaded vs retrieved trade-off to place each fact:

Is this needed on ALMOST EVERY task in this repo?
   YES → PROJECT MEMORY (CLAUDE.md / .cursor/rules / copilot-instructions / AGENTS.md)
         keep it SMALL + STABLE (it's taxed & cached every call)              [§2,§4]
   NO  → is it occasionally relevant, or grows over time?
         YES → LONG-TERM MEMORY (retrieved store / Memories / vector DB)
               store many; inject only the relevant few                       [§3.4]
   Is it only relevant to THIS conversation?
         → leave it in WORKING/SESSION memory (don't persist it)

Cost & reliability implications (straight from what-happens):

• Project-memory bloat ⇒ higher cost every call and worse "lost-in-the-middle" recall (§4). Trim ruthlessly.
• Volatile content in project memory ⇒ cache busts (§2). Keep it stable.
• Over-reliance on retrieval ⇒ silent misses. Keep the truly universal rules always-loaded.

8. Best practices

• Decide delivery deliberately: always-loaded for universal rules, retrieved for the long tail. Don't dump everything into CLAUDE.md.
• Keep project memory small, structured, and stable — it rides every call and (if stable) gets cached.
• Index + retrieve for long-term memory: a tiny always-loaded index of one-liners + retrieved bodies gives awareness without the token tax.
• Don't persist conversation-only details as long-term facts — that's memory pollution; future recalls drown in noise.
• Re-state critical constraints after compaction — a /compact can drop a nuance that wasn't in project memory (§3.1).
• Verify a recalled memory before acting — it reflects what was true when written; if it names a file/flag, confirm it still exists.
• Converge on AGENTS.md where your tools support it, to avoid maintaining parallel instruction files.

The takeaway: there is no memory inside the model. "Project memory," "rules," "context," and "long-term memory" are four points on two axes — lifespan and delivery — describing what text the harness chooses to put back into a stateless model's context. Get the axes right and the buzzwords resolve into clear engineering choices.

📚 References

Primary tool docs

• Claude Code — memory & CLAUDE.md — https://docs.anthropic.com/en/docs/claude-code/memory
• Cursor — Rules — https://docs.cursor.com/context/rules
• GitHub Copilot — custom instructions — https://docs.github.com/en/copilot/customizing-copilot
• BMAD-METHOD — https://github.com/bmadcode/BMAD-METHOD
• AGENTS.md — https://agents.md/

Memory frameworks

• Letta (MemGPT) — https://github.com/letta-ai/letta · paper: https://arxiv.org/abs/2310.08560
• mem0 — https://github.com/mem0ai/mem0

Mechanism deep-dives in this curriculum

• The lifecycle, context, caching, reduction, resume — What Happens When You Prompt an LLM Agent
• Stateless model + Six Laws — Phase 1.00
• Retrieval / RAG — Phase 9
• Agent loop & tools — Phase 10

Up: References · Companion: What Happens When You Prompt an LLM Agent