« Phase 06 README · Warmup · Hitchhiker's · Deep Dive · Principal Deep Dive · Staff Notes

Phase 06 — Core Contributor Notes: Advanced Retrieval (GraphRAG & RAPTOR)

This is the maintainer's-eye view of the real systems our miniature stands in for: Microsoft's GraphRAG library, the RAPTOR reference implementation, and LightRAG. What actually happens inside the pipelines, which design decisions are non-obvious, where the sharp edges are, and what our stdlib version deliberately simplifies. Where I am describing a pattern rather than a specific constant, I say so — do not quote me magic numbers a committer would know to check against a current config.

Microsoft GraphRAG: the indexing pipeline is the product

The thing to internalize about Microsoft GraphRAG is that it is not "a retriever" — it is an indexing pipeline whose output is a set of on-disk artifacts, and the query engine is a thin layer over those artifacts. The pipeline runs as a sequence of stages, each consuming the previous stage's table and emitting a new one, persisted as Parquet files (entities, relationships, communities, community reports, text units, and the covariate/claim tables). This is a real design commitment: the artifacts are the interface. You can rebuild the query layer, swap the store, or inspect the graph in a notebook because every intermediate is a materialized table, not hidden state. When you debug a bad GraphRAG answer as a maintainer, you open the community-reports Parquet and read the summary that produced it — the artifact boundary is your debugging surface.

The stages, in the order they run:

1. Text unit chunking — documents split into text units (the retrieval/extraction granularity).
2. Entity & relationship extraction — the expensive LLM stage. A prompt asks the model to emit typed entities and relationships from each text unit. This is where the cost lives and where the quality ceiling is set.
3. Gleanings — the non-obvious one. After the first extraction pass, GraphRAG re-prompts the model ("MANY entities were missed — add them") for a bounded number of additional rounds. This materially raises recall and materially raises cost; the max-gleaning count is a tuning dial that trades tokens for completeness. Our miniature has no analogue — the injected extractor returns everything in one deterministic call.
4. Graph construction & entity summarization — merge duplicate entities, summarize entity descriptions across the mentions.
5. Community detection — hierarchical Leiden, producing communities at multiple levels (level 0 coarse, deeper levels finer). This is the graph analogue of RAPTOR's tree: theme granularity you can summarize at several resolutions.
6. Community report generation — one LLM-written report per community per level. These reports are what global_query reduces over. They are the feature.
7. Embedding — entities, text units, and reports get embedded for the query engine.

Global vs local search in the real library map onto what we built. Global search is a map-reduce over community reports at a chosen Leiden level: map the question over each report to produce partial answers with helpfulness scores, then reduce the top partials into a final answer. Local search starts from entities matched to the query, expands to their neighbors, connected text units, and covariates, and assembles a context window. Our global_query is the retrieval core of the map step; our local_query is the entity-neighborhood expansion without the text-unit join.

Prompt tuning is a first-class step. GraphRAG ships an auto prompt-tuning command that reads a sample of your corpus and generates domain-adapted extraction prompts and entity-type lists. This exists because the default entity types (person, organization, geo, event) are wrong for a fraud or infrastructure corpus, and extraction quality collapses if the entity taxonomy does not match the domain. A committer knows: the first thing you do on a new corpus is tune the prompts and the entity-type config, not run the default pipeline and complain about the graph.

RAPTOR reference implementation: UMAP + GMM + BIC

The RAPTOR paper's clustering is more sophisticated than our greedy scheme, and the three pieces are worth naming precisely because they are the exam questions.

• UMAP reduces the embedding dimensionality before clustering. High-dimensional embeddings make density-based clustering ineffective (distances concentrate); UMAP projects to a low-dimensional space where a mixture model can find structure. The paper does this in two resolutions — a global UMAP to find broad clusters, then a local UMAP within each global cluster to find sub-clusters.
• Gaussian Mixture Model (GMM) does soft clustering: each node gets a probability of membership in each cluster, so a node above a threshold can belong to several clusters — a chunk bridging two topics is summarized into both parents. Our greedy hard-clustering cannot do this; it assigns each node to exactly one cluster. That is the single biggest fidelity gap in the miniature, and it is deliberate — soft membership needs a seed and is painful to make byte-reproducible.
• BIC (Bayesian Information Criterion) selects the number of clusters. Rather than fixing cluster_size as we do, the reference fits GMMs with varying component counts and picks the one minimizing BIC — it discovers how many clusters the data wants. Our miniature fixes the branching factor; the real one infers it.

Retrieval in the reference is the collapsed tree (they call it "collapsed tree retrieval"): flatten all nodes across levels into one pool and take top-k by similarity, the same thing our collapsed_tree_search does. The paper reports it matching or beating tree-traversal, which is why the collapsed mode is the sensible production default. The summaries in the reference are real LLM summaries; ours is the frequency-word extractive stand-in, faithful only in that a summary covers the union of its children's terms.

LightRAG: dual-level keys and incremental insert

LightRAG's two contributions are the dual-level retrieval keying and the incremental graph update.

The dual-level keying is the design decision people miss. LightRAG extracts, for each query, low-level keys (specific entities/nouns) and high-level keys (broad themes/concepts), and retrieves against both — low-level keys hit entity neighborhoods, high-level keys hit the theme descriptions. Our split of local_query (entity-anchored) and global_query (theme-anchored) is exactly this dichotomy, minus the automatic key extraction from the query.

The incremental insert is the operational reason LightRAG exists. New documents are extracted, their entities/edges merged into the existing graph (deduplicating against existing nodes), and only the touched regions re-summarized — no global Leiden re-partition, no full report regeneration. This is the direct answer to full GraphRAG's re-index problem. LightRAG also leans on a key-value/graph store combination rather than a heavyweight hierarchical community pass, which is why it advertises far fewer LLM calls to index and update. The tradeoff is a flatter thematic structure than Leiden's multi-level hierarchy — you get local + global, not global-coarse + global-fine + …

Sharp edges a committer knows

• Extraction prompt sensitivity. The single most impactful surface. Small prompt wording changes swing which entities and relations get extracted, which swings the entire downstream graph. This is why GraphRAG made prompt tuning a command and why "the graph looks wrong" almost always traces to the extraction prompt or entity-type config, not the algorithm.
• Gleanings are a cost multiplier. Each gleaning round is another full LLM pass over every chunk. Recall goes up, the bill goes up linearly with the round count. Tuning this dial is a real cost/quality decision, not a default to leave alone.
• Token blowup at community-report time. A large community has many relationships; rendering them all into the report prompt can blow the context window, so the real systems rank/trim relationships by importance before summarizing. Our miniature renders every triple — fine for a handful, a context-overflow bug at scale.
• Entity resolution is where the tuning lives. Stripe / stripe_gateway / the gateway must collapse to one node or the graph fragments. This is unglamorous, corpus-specific, and it is the difference between a graph that answers multi-hop questions and one that quietly returns garbage.
• Determinism is not free. Leiden has a random seed; UMAP+GMM have seeds; LLM extraction is nondeterministic even at temperature zero across model versions. Reproducing a graph exactly across runs is genuinely hard, which is why our miniature injects the nondeterministic parts and uses connected components and greedy clustering instead.

What our stdlib miniature deliberately simplifies

The miniature is faithful in control flow and retrieval idea, and honest about what it drops:

• Greedy hard-clustering instead of UMAP+GMM+BIC soft clustering — no multi-parent membership, fixed branching instead of discovered cluster count.
• Connected components instead of hierarchical Leiden — exact for disjoint themes, cannot split a connected blob into sub-communities or produce a multi-level community hierarchy.
• A frequency-word extractive summarizer instead of an LLM — reproducible, and faithful only in that a summary covers the union of its inputs' terms (which is enough to make the RAPTOR inequality hold and the community summaries win a global query).
• An injected deterministic extractor instead of an LLM with gleanings and entity resolution — so the graph is a pure function of the corpus and the tests assert exact nodes and edges. No prompt sensitivity, no gleaning cost, no resolution ambiguity — precisely the parts that make the real systems expensive and hard, removed on purpose so the algorithm is visible.

The load-bearing claim of the lab is that the algorithm — collapse the tree, detect communities, summarize them, split local from global — is identical whether the vector came from a hash or a transformer and whether the summary came from a Counter or an LLM. The expensive, nondeterministic machinery the real systems wrap around that algorithm is what the Principal Deep Dive prices and the Staff Notes decide whether to pay for.