« Phase 06 README · Hitchhiker's · Deep Dive · Principal Deep Dive · Core Contributor · Staff Notes

Phase 06 Warmup — Advanced Retrieval: GraphRAG, LightRAG & RAPTOR

Who this is for: you did Phase 05 and can chunk, embed, and run a hybrid BM25 + dense retriever with RRF and a reranker. You have never built a retriever that answers a question no single chunk contains — a synthesis, a global theme, a multi-hop traversal. By the end you will have built two: a RAPTOR recursive-summary tree and a mini-GraphRAG, and you will know exactly which question shape wants which. No GPU, no API key, no vector DB — the embedder, summarizer, and extractor are functions you inject.

Table of Contents

1. Why flat vector RAG hits a wall
2. Three failure shapes and their named fixes
3. The embedder and cosine, recapped from Phase 05
4. RAPTOR: the recursive cluster-and-summarize idea
5. Clustering nodes: real soft clustering vs our deterministic greedy
6. Building the tree: summaries as first-class nodes
7. Collapsed-tree vs tree-traversal retrieval
8. Why a summary node beats a leaf on a synthesis query
9. GraphRAG: from a pile of chunks to a knowledge graph
10. Entity and relation extraction: the expensive index pass
11. Community detection: components, label propagation, and Leiden
12. Community summaries and the map-reduce for global queries
13. Local vs global: LightRAG dual-level retrieval and incremental updates
14. The stores: Neo4j, pgvector, and hybrid
15. Choosing an architecture: cost, build-time, and the decision
16. Common misconceptions
17. Lab walkthrough
18. Success criteria
19. Interview Q&A
20. References

1. Why flat vector RAG hits a wall

The Phase 05 retriever does one thing: given a query, it returns the k existing chunks whose embeddings are nearest the query embedding. That is exactly right for a fact-lookup question — "what port does the service listen on?" — where some chunk literally contains the answer and your only job is to find it.

Now ask a different kind of question over the same corpus:

• "Summarize how this 90-page architecture doc evolved from v1 to v3."
• "What are the main themes across our last 200 incident retros?"
• "Which team owns the model that flags the payments the gateway processes?"

Run each through top-k dense retrieval and watch it fail in a specific way. The summary query returns a few chunks that each mention "v2" — but the summary of the evolution is written in none of them, so the model gets fragments and hallucinates the arc. The themes query returns 200 chunks that are all "relevant" and none of which say "the themes are X, Y, Z" — because a theme is a property of the whole set, not of any member. The ownership query returns the chunk about the gateway and the chunk about the model, but the path connecting them — the multi-hop relation — is something you have to traverse, and a bag of nearest chunks has no edges to walk.

The one sentence that captures all three: the answer isn't in any single chunk. Flat retrieval can only ever hand you chunks that already exist, so when the answer is a synthesis, a corpus-level property, or a relationship, retrieval-of-existing-text is structurally the wrong tool. This phase is the set of fixes.

Why this is a senior signal. Junior engineers respond to "RAG is giving bad answers" by tuning chunk size and k. The senior move is to notice the question shape — "this is a global/thematic question, no chunk has the answer" — and change the architecture, not the hyperparameters. Saying that out loud is most of the interview.

2. Three failure shapes and their named fixes

Each failure shape from §1 has a named architecture, and the whole phase is the mapping:

Two of these — RAPTOR and GraphRAG — share a deep move: they shift work from query-time to index-time. Flat RAG does nothing clever at index time (just embed the chunks) and hopes query-time nearest-neighbor is enough. RAPTOR pre-computes abstraction (summaries); GraphRAG pre-computes structure (a graph + community summaries). That is why they are more powerful and why their cost lives in the (expensive) indexing pass — a theme we return to in §15.

LightRAG is the third name on Citi's list. It is not a fourth failure shape; it is a cheaper way to get GraphRAG's benefits: dual-level (local + global) retrieval over a graph, with incremental updates so a new document doesn't force a full re-index. We build its local + global split as two query functions in the lab (§13).

3. The embedder and cosine, recapped from Phase 05

Everything here still rests on the Phase 05 primitives, so a 30-second recap. We turn text into a vector with a hashing bag-of-words embedder: tokenize, hash each token to a bucket in [0, dim), count occurrences, and L2-normalize. Similarity is cosine — the dot product of two unit vectors, in \([-1, 1]\) (in \([0,1]\) for our all-nonnegative count vectors):

$$\cos(\mathbf{a},\mathbf{b}) = \frac{\mathbf{a}\cdot\mathbf{b}}{\lVert\mathbf{a}\rVert,\lVert\mathbf{b}\rVert}.$$

Two properties matter for this phase. First, it is deterministic and offline — we hash with hashlib.sha256, never Python's salted built-in hash(), so the same text always yields the same vector across processes, which is what makes the tree, the clusters, and the query winners testable. Second, it is semantics-free: it matches shared tokens, not shared meaning. A real system uses a trained embedder that clusters on meaning; our stand-in matches words, so the lab's corpus is stylized (strong shared "anchor" words per theme) to make the mechanism visible. The algorithms you build — clustering, tree-building, community detection, map-reduce — are identical whether the vector came from a hash or a transformer.

The whole discipline of these labs (see LAB-STANDARD) is inject the non-deterministic part. Here that's three callables: the embedder, the summarizer, and the triple extractor. In production all three are (or wrap) an LLM; injecting them makes the retrieval pipeline a pure function of its inputs.

4. RAPTOR: the recursive cluster-and-summarize idea

RAPTOR — Recursive Abstractive Processing for Tree-Organized Retrieval (Sarthi et al., 2024) — fixes the synthesis failure by building a tree of summaries at index time. The algorithm is a loop:

1. Start with the leaf chunks (level 0) and embed them.
2. Cluster the current level's nodes into groups of similar nodes.
3. Summarize each cluster into a single parent node (an LLM writes the summary), and embed the parent.
4. The parents become the next level. Recurse from step 2 until one node remains — the root.

        [ root: whole-corpus gist ]              level 2
         /            |            \
   [sum A]        [sum B]        [sum C]         level 1  (cluster summaries)
   / | \          / | \          /   \
 c0 c1 c2       c3 c4 c5       c6    c7          level 0  (original chunks / leaves)

Each level up is more abstract and covers more source text. A level-1 node is the summary of one cluster of chunks; the root is a summary-of-summaries covering everything. The key insight is that a summary node is a legitimate retrieval target — it is embedded and stored right alongside the leaves, so a query can match it directly.

Why does this answer synthesis? Because the expensive act of combining several chunks into one coherent statement has already happened, at index time, for many natural groupings. When a synthesis query arrives, retrieval doesn't have to fuse chunks on the fly (it can't — it only ranks) — it just finds the pre-fused summary node that already did the work. We prove this happens, and quantify why, in §8.

5. Clustering nodes: real soft clustering vs our deterministic greedy

RAPTOR's step 2 needs to group similar nodes. The paper does this with soft clustering: project embeddings down with UMAP, then fit a Gaussian Mixture Model so a node can belong to several clusters (a chunk that bridges two topics gets summarized into both parents). That is powerful and correct — and it needs a random seed, an ML library, and a dimensionality reduction that is painful to make byte-reproducible.

For the lab we use a deterministic greedy nearest-neighbor clustering that captures the same idea — put each node with the ones most like it — with none of the nondeterminism:

• Seed nodes in a fixed order (sorted id).
• Each seed pulls its cluster_size - 1 nearest still-unclustered neighbors by cosine (ties broken by id).
• Repeat until every node is assigned.

Two invariants make this safe as a tree-builder. Because cluster_size >= 2, every cluster is strictly smaller than the input whenever there is more than one node — so the number of nodes strictly decreases each level and the tree provably terminates at a single root. And because every step is a pure function of the embeddings and the sorted ids, the clustering is deterministic: same corpus, same tree, every run. A test asserts both (greedy_cluster returns identical output twice, and the same-theme chunks land together).

The honest tradeoff. Greedy hard-clustering can't put one node in two parents, and it fixes cluster size instead of discovering it. Real RAPTOR's GMM does both. We trade that fidelity for reproducibility — the right call for a teaching lab, and exactly the kind of "here's what the miniature gives up" you state in an interview.

6. Building the tree: summaries as first-class nodes

The tree builder is the loop from §4 made concrete. build_raptor_tree(chunks, embed, summarizer, cluster_size) creates one RaptorNode per chunk at level 0, then while the current level has more than one node: cluster it, summarize each cluster's texts into a parent node (recording the parent's children so the tree is navigable), embed the parent, and ascend. The result is a RaptorTree holding every node keyed by id, a levels map (level → node ids), and the root_id.

The subtle, important design choice: the parent's text is the summarizer's output, and we embed that. The summary is not metadata bolted onto a chunk; it is a full node with its own embedding, indistinguishable at retrieval time from a leaf except by its level. That is what lets a single flat search rank leaves and summaries against each other (§7).

In the lab the injected summarizer is a deterministic extractive stand-in (make_frequency_summarizer): it keeps the most frequent content words across the cluster's texts. A real summarizer is an LLM ("write a 3-sentence summary of these chunks"). The stand-in is faithful in the one way that matters for retrieval: a summary of several chunks covers the union of their key terms, which is precisely the property that makes it win a synthesis query.

7. Collapsed-tree vs tree-traversal retrieval

Once the tree exists, RAPTOR offers two retrieval modes, and knowing the difference is a classic interview probe.

Tree traversal. Start at the root, score its children against the query, descend into the top-scoring child (or top-few), and repeat down the levels, collecting nodes as you go. It reads the tree as a tree. It visits few nodes (log-depth), but an early wrong turn at a high level prunes away a leaf you needed.

Collapsed tree. Ignore the hierarchy for retrieval: pour every node at every level into one flat pool and take the top-k by cosine, exactly like a normal vector search — except the pool now contains summaries as well as leaves. It scores more nodes (all of them) but never prunes, so a leaf detail and a high-level summary compete head-to-head and the query decides which wins.

  tree-traversal:  root ─► best child ─► best grandchild ...   (few nodes, can mis-prune)
  collapsed-tree:  { all leaves } ∪ { all summaries }  ─► top-k by cosine   (no pruning)

Sarthi et al. found the collapsed tree matches or beats tree traversal on QuALITY and NarrativeQA, because it lets the query pick its own level of abstraction: a specific question naturally lands on a leaf, a synthesis question naturally lands on a summary — no routing logic required. That is why the lab implements collapsed-tree search as the headline (collapsed_tree_search) and leaves tree traversal as an extension.

8. Why a summary node beats a leaf on a synthesis query

Here is the mechanism, made quantitative, because "the summary wins" should be math you can derive, not magic you assert. Suppose a cluster has \(n\) leaves, and each leaf \(i\) contributes one distinctive term \(t_i\) (its own fact) on top of shared filler. The cluster's summary covers the union, so it contains all of \(t_1,\dots,t_n\).

Now a synthesis query asks about the combination — its text mentions each distinctive fact, so its bag-of-words contains \(t_1,\dots,t_n\). Compare cosines (ignoring shared filler, which helps every candidate equally):

• Query vs leaf \(j\): the only distinctive term they share is \(t_j\), so the dot product is \(\approx 1\). With \(\lVert q\rVert \approx \sqrt{n}\) and a short leaf, the cosine is small.
• Query vs summary: they share all \(n\) distinctive terms, so the dot product is \(\approx n\). The summary is longer (norm grows like \(\sqrt{n + \text{extras}}\)), but:

$$\cos(q,\text{summary}) \approx \frac{n}{\sqrt{n},\sqrt{n+\text{extras}}} = \sqrt{\frac{n}{n+\text{extras}}}, \qquad \cos(q,\text{leaf}_j) \approx \frac{1}{\sqrt{n},\lVert \text{leaf}_j\rVert}.$$

For \(n \ge 2\) the summary's numerator advantage (\(n\) vs \(1\)) dominates its length penalty, so the summary node outscores every single leaf. That is the whole RAPTOR payoff in one inequality: a query that needs several chunks lands on the node that already fused them.

The lab makes this concrete and tests it. The corpus has a three-chunk "Mars" theme (each chunk one distinctive fact: perseverance, ingenuity, orbiter), and the synthesis query names all three. collapsed_tree_search returns the summary node (level 1, cosine ≈ 0.68) above the best leaf (cosine ≈ 0.34), while flat_leaf_search — the Phase 05 baseline — can only return leaves. Same query, and only the tree with summary nodes can answer it. (This is also why the lab's corpus is stylized: with a semantics-free embedder we make the distinctive terms literal so the mechanism is visible; a real embedder gets the same effect from meaning.)

9. GraphRAG: from a pile of chunks to a knowledge graph

RAPTOR abstracts within a document's content. GraphRAG (Edge et al., Microsoft, 2024) attacks the global/thematic and multi-hop failures by imposing structure on the whole corpus: it builds a knowledge graph and reasons over it.

A knowledge graph is entities as nodes and relations as edges. The unit is the triple (subject, relation, object) — e.g. (fraud_model, flags, payments). Extract enough triples from a corpus and you get a graph where you can do things a bag of chunks cannot:

• Traverse — follow edges to answer multi-hop questions ("who owns the model that flags the payments?" is a two-hop walk payments ← flags ← fraud_model ← owns ← risk_team).
• Find structure — dense clusters of interconnected entities are communities (§11), and a community is a theme the corpus is "about," even though no chunk names it.

The GraphRAG pipeline is four index-time stages plus two query modes:

  INDEX:  extract triples ─► build graph ─► detect communities ─► summarize each community
  QUERY:  global = map-reduce over community summaries   |   local = walk an entity neighborhood

The lab builds all of it in miniature: extract_graph (stage 1–2), detect_communities (stage 3), summarize_communities (stage 4), global_query and local_query (the two modes).

10. Entity and relation extraction: the expensive index pass

Stage one turns text into triples. In production this is an LLM prompted to emit (entity, relation, entity) tuples for each chunk ("extract all entities and their relationships as JSON triples"), often with a second pass to resolve duplicates (Stripe, stripe_gateway, and the gateway are one entity). This is the cost center of GraphRAG: one-or-more LLM calls per chunk, across the entire corpus, before you can answer a single question. Indexing a large corpus can be thousands of dollars and hours of LLM time — which is the number you must bring to the design review (§15).

In the lab the extractor is injected: extract_graph(docs, extractor) calls extractor(doc) -> list[(subject, relation, object)] and adds each triple to a KnowledgeGraph, which maintains the entity set, the triple list, and an undirected adjacency map (community detection treats a relation as a connection regardless of direction). Injecting the extractor makes the graph a deterministic function of the corpus, so a test can assert the exact nodes and edges — and it mirrors how you'd unit-test a real extraction pipeline (fixture in, known graph out) without paying for LLM calls in CI.

Extraction quality is the ceiling. Everything downstream — communities, summaries, traversals — inherits the extractor's mistakes. A missed relation is an edge that isn't there, a hallucinated one is a false path. In production, extraction prompt quality and entity resolution are where most of the GraphRAG tuning effort actually goes.

11. Community detection: components, label propagation, and Leiden

Stage three finds communities — groups of densely interconnected entities that represent a theme. There's a spectrum of algorithms, and you should be able to place them:

• Connected components — the simplest: two entities are in the same community iff a path of edges connects them. Deterministic, exact when themes form disjoint subgraphs, but it can't split a single big connected blob into sub-themes. This is what the lab implements (detect_communities, via sorted-order BFS so it's reproducible), and it's the right stand-in when your corpus separates cleanly.
• Label propagation — each node adopts the majority label of its neighbors, iterated to a fixed point. Cheap and can split within a component, but needs care (tie-breaking, update order) to be deterministic.
• Leiden (and its predecessor Louvain) — the production choice, used by Microsoft GraphRAG. It maximizes modularity (edges-inside-community minus what you'd expect at random) and, unlike Louvain, guarantees well-connected communities. It is hierarchical: it finds communities, then communities-of-communities, giving you theme granularity to summarize at multiple levels — the graph analogue of RAPTOR's tree.

The lab uses connected components because it is deterministic and exact for a well-separated corpus; the extension swaps in networkx's Leiden/Louvain to split sub-communities within a component. The idea you need to defend is identical: group related entities so each group is a summarizable theme.

12. Community summaries and the map-reduce for global queries

Stage four is the one people miss, and it is the entire point of GraphRAG. For each community, gather the relations among its members and have the LLM write a community summary — a paragraph describing what this cluster of entities is about. In the lab, summarize_communities renders each community's triples as short "subject relation object" sentences and feeds them to the injected summarizer, producing one CommunitySummary per community.

These summaries are how you answer a global question. A global query — "what are the main themes?" — is answered by a map-reduce over communities:

• Map: ask the question against each community summary independently (each produces a partial answer, "this community is about payment-fraud detection").
• Reduce: combine the partial answers into a final response.

The lab's global_query(query, community_summaries, embed, k) is the retrieval core of that: embed the query and each community summary, return the top-k communities by cosine. For a whole-corpus question you set k to the number of communities (the full map-reduce); for a themed question, k=1 finds the matching community. The lab proves it: a query about "which model flags fraud payments and which team owns the risk model" returns the payments/fraud community summary (cosine ≈ 0.6) over the cloud-infra one (≈ 0.25) — even though no source chunk contains the phrase "the themes are fraud detection and infrastructure." The community summary is where that corpus-level knowledge now lives.

This is the misconception to kill. GraphRAG is not "vector RAG with the vectors stored in a graph database." Storing embeddings in Neo4j buys you nothing new. GraphRAG's power is the community summaries — new text, written at index time, that states themes no chunk states. If you remember one thing from this phase, remember that.

13. Local vs global: LightRAG dual-level retrieval and incremental updates

LightRAG (Guo et al., 2024) is the third Citi acronym, and it is best understood as "GraphRAG's benefits, cheaper." Two ideas:

Dual-level retrieval. LightRAG splits queries into two key types and retrieves for both:

• Low-level / local keys — specific entities and their neighbors. "Tell me about fraud_model and what it connects to" is answered by walking that entity's neighborhood in the graph. This is precise, entity-anchored, and cheap. The lab's local_query(entity, graph) returns exactly this: the entity's 1-hop neighbors and every triple it participates in (as subject or object — you catch incoming edges too).
• High-level / global keys — broad themes, answered from community-level summaries, like GraphRAG's global mode. The lab's global_query is this half.

Together, local_query and global_query are LightRAG's dual-level retrieval — a local "what is X connected to" and a global "what are the themes" over the same graph.

Incremental updates. Full GraphRAG's Leiden + community-summary pass is a global computation: strictly, a new document can change the community structure, so the cautious move is to re-index. LightRAG is designed so a new document merges its new entities/edges into the existing graph and only re-summarizes the affected neighborhoods, avoiding a full rebuild. For a corpus that grows daily, that incremental path is the difference between a viable system and a nightly re-index you can't afford. (The lab flags an add_document incremental-update function as an extension.)

Net: LightRAG typically needs far fewer LLM calls to index and update than full GraphRAG, trading some of GraphRAG's hierarchical-community richness for cost and freshness. "Do we need full GraphRAG or is LightRAG enough?" is a real design-review question, and the answer is a cost argument (§15).

14. The stores: Neo4j, pgvector, and hybrid

Citi's JD pairs the acronyms with two databases — "Integrate graph and vector databases such as Neo4j and pgvector" — because advanced retrieval needs both kinds of index:

• Vector store (pgvector / Pinecone / Weaviate / FAISS / Chroma) — holds embeddings and does approximate-nearest-neighbor search. In RAPTOR it holds both the leaf-chunk and the summary-node embeddings, so collapsed-tree search is one ANN query over the combined pool. pgvector is the JD's named choice: it's a Postgres extension, so your vectors live in the same transactional database as your application rows — no separate system to operate, and you can filter by SQL metadata and vector-similarity in one query.
• Graph store (Neo4j) — holds entities as nodes and relations as edges, and runs graph queries in Cypher (MATCH (e)-[r]-(n)). In GraphRAG/LightRAG it holds the extracted graph, the community assignments, and the traversals local_query performs. Neo4j ships neo4j-graphrag to wire extraction, community detection, and retrieval to an LLM.

Real systems are hybrid: the graph store answers traversal and community questions; the vector store answers similarity and RAPTOR questions; and they're joined on entity/chunk ids. The lab models both stores in memory — RaptorTree is your vector index over nodes; KnowledgeGraph (entities, triples, adjacency) is your graph store — so you see the two shapes of index side by side. Choosing which store answers which query is the integration skill the JD is naming.

15. Choosing an architecture: cost, build-time, and the decision

Assemble it into the decision you'll defend in a review. The ladder, cheapest to most powerful:

Three rules of thumb:

1. Default to flat vector RAG. It's cheap and it's right for most questions. Climb the ladder only when the question shape (§2) demands it — a synthesis, a theme, a traversal.
2. The cost is the index, not the query. RAPTOR and especially GraphRAG move work to index-time: an LLM call per cluster (RAPTOR) or per chunk plus community summaries (GraphRAG). For a large corpus GraphRAG indexing is a real budget line — potentially thousands of LLM calls — so you price it before you commit, using the Phase 00 cost lens ($/index and $/query, not just latency).
3. Prefer LightRAG when the corpus changes. If documents arrive continuously, full GraphRAG's re-index is often infeasible; LightRAG's incremental updates and dual-level retrieval get you most of the benefit at a fraction of the (re)indexing cost.

The Staff-level answer to "should we use GraphRAG?" is rarely "yes, it's better." It's "what's the question shape, how big and how fresh is the corpus, and what's the indexing bill — because for a static corpus with global questions, GraphRAG earns its cost, and for a growing corpus of local questions, LightRAG or plain RAPTOR wins."

16. Common misconceptions

• "GraphRAG is just RAG with a graph database." No — the power is the community summaries that answer global questions, not the storage engine. Putting embeddings in Neo4j is still vector RAG. (§12.)
• "RAPTOR is just chunk summaries." It's a recursive tree of summaries with collapsed- tree retrieval, so a query picks its own abstraction level. A single flat layer of summaries misses the multi-level abstraction and the head-to-head leaf-vs-summary ranking. (§7–§8.)
• "Advanced retrieval is strictly better, so always use it." It's situationally better and always more expensive to index. Flat vector RAG is the right default; the skill is knowing when the question shape justifies climbing the ladder. (§15.)
• "The cost is at query time." The cost is at index time — LLM summaries (RAPTOR) and extraction + community reports (GraphRAG). A global query is cheap because the expensive pre-computation already ran. (§10, §15.)
• "LightRAG is a worse GraphRAG." It's a different point on the cost curve: dual-level retrieval + incremental updates, chosen when cost and freshness matter more than hierarchical- community richness. (§13.)
• "Community detection needs Leiden or it doesn't count." Leiden is the production choice for splitting a connected graph into hierarchical themes, but connected components is exact for a well-separated corpus. The idea — group related entities into summarizable themes — is what matters. (§11.)
• "More tree levels / more communities = better." Past the point where a level or a community is a coherent theme, extra structure is index cost and dilution, not signal. Depth is a parameter to justify, not maximize.

17. Lab walkthrough

Open lab-01-raptor-graphrag/ and fill the TODOs top to bottom — the file is ordered to match this warmup:

1. embed + cosine — the Phase 05 primitives: hash tokens to buckets, count, L2-normalize; cosine with a zero-vector guard. Everything downstream depends on these.
2. greedy_cluster — the deterministic nearest-neighbor grouping (§5). Sorted-id seeds, nearest neighbors by cosine, ties by id. A test checks determinism and that same-theme chunks group.
3. build_raptor_tree — level 0 leaves, then cluster → summarize → ascend to a single root (§6). Record each parent's children. Tests check strictly-decreasing level counts and a single root.
4. collapsed_tree_search (and the flat_leaf_search baseline) — score all nodes / only leaves, top-k by cosine (§7). The headline test: on the synthesis query, collapsed search ranks a summary node above any leaf (§8).
5. extract_graph — run the injected extractor, build the KnowledgeGraph (§10).
6. detect_communities — deterministic connected components via sorted BFS (§11).
7. summarize_communities + global_query — community summaries and the map-reduce retrieval that answers a global question (§12).
8. local_query — the entity neighborhood: LightRAG's local retrieval (§13).

Run LAB_MODULE=solution pytest test_lab.py -v first to see green (33 tests), then make your lab.py match. Finish by reading solution.py's main() output — it builds the RAPTOR tree, shows the summary node beating the leaves on a synthesis query, then builds the graph, detects communities, and answers a global thematic query and a local entity query.

18. Success criteria

• You can name the three question shapes that break flat vector RAG and the architecture that fixes each — and recognize a shape from a fresh question.
• You can explain RAPTOR's build loop and why the collapsed tree beats tree traversal, and derive (from §8's math) why a summary node wins a synthesis query.
• You can walk the GraphRAG pipeline end to end and say why community summaries answer a global question that no chunk contains — and why that, not the graph DB, is the point.
• You can state LightRAG's two changes (dual-level retrieval, incremental updates) and where Neo4j vs pgvector each fit.
• You can price the architectures (index cost vs query cost) and defend a choice.
• All 33 lab tests pass under both lab and solution.

19. Interview Q&A

Q: Vector RAG is returning bad answers for "summarize the incident trends this quarter." What's wrong and what do you do? A: Wrong architecture, not wrong hyperparameters. That's a global/synthesis question — the answer isn't in any single chunk, so top-k over chunks can only return fragments. I'd reach for RAPTOR (if it's synthesis over a bounded set of docs — retrieve the pre-computed summary node) or GraphRAG (if it's a theme over the whole corpus — build the incident graph, detect communities, and map-reduce the community summaries). The tell is "no single chunk has the answer."

Q: Explain RAPTOR, and what "collapsed tree" means. A: RAPTOR recursively clusters chunks and summarizes each cluster into a parent node, building a tree where higher levels are more abstract and cover more text — summaries are embedded and stored as first-class retrieval targets. "Collapsed tree" is a retrieval mode: instead of traversing the tree top-down (which can mis-prune at a high level), you pour all nodes at all levels into one pool and take the top-k by similarity. A specific query lands on a leaf; a synthesis query lands on a summary — the query picks its own abstraction level, no routing needed. The paper shows collapsed-tree matches or beats traversal.

Q: Why does a summary node beat a leaf on a synthesis query — concretely? A: The summary covers the union of its children's distinctive terms; a synthesis query mentions several of those terms. So the query shares one distinctive term with any single leaf but all n with the summary — dot product ~1 vs ~n. The summary is longer (bigger norm), but for n ≥ 2 the n-vs-1 numerator advantage wins, so its cosine is higher. It's the node that already fused the chunks the query needs together.

Q: What actually makes GraphRAG different from vector RAG — is it the graph database? A: No. The database is incidental. GraphRAG's difference is the community summaries: at index time it extracts an entity/relation graph, detects communities (Leiden) of related entities, and has an LLM write a summary of each community. Those summaries are new text stating corpus-level themes that appear in no source chunk, and a global query is a map-reduce over them. Store embeddings in Neo4j and you've still got vector RAG; write community summaries and you've got GraphRAG.

Q: When would you pick LightRAG over GraphRAG? A: When cost and freshness matter more than hierarchical-community richness. LightRAG does dual-level (local entity-neighborhood + global theme) retrieval and, crucially, incremental graph updates — a new document merges in and only re-summarizes affected neighborhoods instead of forcing a full Leiden + community-report re-index. For a corpus that grows daily, that's the difference between viable and unaffordable. Full GraphRAG earns its heavier indexing on a static corpus with lots of global questions.

Q: A stakeholder wants GraphRAG "because it's state of the art." How do you scope it? A: I'd ask three things: the question shapes (are they global/multi-hop, or mostly fact lookup — if the latter, we don't need it), the corpus size and freshness (extraction is one-or-more LLM calls per chunk, so I'd estimate the indexing bill and the re-index cadence), and the store fit (Neo4j for traversal + pgvector for similarity). Often the answer is RAPTOR for synthesis or LightRAG for a growing corpus — most of the benefit, a fraction of the index cost. GraphRAG is the right call for a static corpus of genuinely global questions, and I'd bring the cost number.

Q: How do Neo4j and pgvector split the work in a hybrid system? A: pgvector (or another vector store) holds embeddings — chunk and RAPTOR summary nodes — and answers similarity / collapsed-tree queries with ANN; being a Postgres extension it also gives you SQL metadata filtering in the same query. Neo4j holds the entity/relation graph and community assignments and answers traversals (Cypher MATCH) and local neighborhood queries. They're joined on entity and chunk ids, and you route each query to the store whose index shape fits it.

20. References

• Sarthi et al., RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (ICLR 2024). https://arxiv.org/abs/2401.18059
• Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Microsoft, 2024). https://arxiv.org/abs/2404.16130
• Guo et al., LightRAG: Simple and Fast Retrieval-Augmented Generation (2024). https://arxiv.org/abs/2410.05779
• Microsoft GraphRAG — project & docs. https://microsoft.github.io/graphrag/
• Traag, Waltman & van Eck, From Louvain to Leiden: guaranteeing well-connected communities (2019). https://arxiv.org/abs/1810.08473
• Neo4j GraphRAG for Python (neo4j-graphrag) — docs. https://neo4j.com/docs/neo4j-graphrag-python/current/
• pgvector — open-source vector similarity search for Postgres. https://github.com/pgvector/pgvector
• Gao et al., Retrieval-Augmented Generation for Large Language Models: A Survey (2023) — the RAG landscape this phase sits in. https://arxiv.org/abs/2312.10997
• LlamaIndex RAPTOR pack & LangChain RAPTOR cookbook — reference implementations. https://docs.llamaindex.ai/ · https://github.com/langchain-ai/langchain