Retrieval & Context

GraphRAG

GraphRAG is a retrieval-augmented generation approach that builds an LLM-extracted knowledge graph of entities and relationships over a corpus, then answers questions by traversing that graph and summarizing graph communities, rather than retrieving only flat vector chunks. It targets multi-hop and global "sensemaking" questions that ordinary chunk-based RAG handles poorly.

What GraphRAG is

GraphRAG is a family of retrieval-augmented generation (RAG) methods that ground a language model's answer in a knowledge graph built from a corpus, instead of, or alongside, a flat set of embedded text chunks. In ordinary RAG, documents are split into passages, embedded as vectors, and the top passages most similar to the query are concatenated into the prompt. GraphRAG adds a structured layer: an indexing phase uses a language model to extract entities (people, places, concepts) and the relationships between them, assembling a graph whose nodes are entities and whose edges are typed relationships.

The term is most closely associated with the Microsoft Research approach described in the 2024 paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (arXiv:2404.16130) and the accompanying open-source release. That work frames GraphRAG as a way to answer broad questions about an entire dataset, the kind of query where no single passage contains the answer and where naive top-k chunk retrieval returns fragments that miss the whole picture.

GraphRAG is best understood as a two-stage system. The expensive, offline index stage turns unstructured text into a graph plus a set of precomputed community summaries. The cheaper, online query stage routes a question through that structure, either drilling into a local neighborhood of entities or aggregating across high-level community summaries for a global answer. The graph is a complement to vector retrieval, not always a full replacement; many implementations keep embeddings for entities and text units and use the graph for structure and reach.

Nodes are entities and edges are relationships, both extracted from text by an LLM during indexing.
It targets multi-hop and corpus-wide questions, not just "find the most similar passage."
It pairs an offline graph-and-summary build with an online graph-aware query stage.
It can sit alongside vector search rather than fully replacing embeddings.

Building the graph: entity and relationship extraction

Indexing begins by splitting source documents into text units, then prompting a language model to read each unit and output structured records: the entities it mentions, short descriptions of them, and the relationships connecting them. Because the same entity is described differently across many text units, descriptions for repeated entities and edges are merged, so a single node accumulates evidence from across the corpus. The result is a typed graph rather than an undifferentiated bag of chunks.

Using an LLM as the extractor is what distinguishes GraphRAG from earlier knowledge-graph pipelines that relied on hand-built schemas and brittle rule-based or supervised named-entity-recognition systems. The LLM can extract entities and relations from arbitrary domains with only a prompt, which makes graph construction far cheaper to set up, at the cost of variability and the need for prompt tuning per corpus. This trade is central to GraphRAG: graph quality is bounded by extraction quality.

Extraction is the most token-intensive and error-prone part of the pipeline. Every text unit is sent to a model, so indexing cost scales with corpus size, and the official tooling explicitly warns that indexing can be expensive and recommends tuning the extraction prompts before a production run. Inconsistent entity naming, missed relationships, and hallucinated edges all degrade downstream retrieval, which is why deduplication and description summarization are built into the index.

Documents are chunked into text units, each read by an LLM to emit entities and relationships.
Repeated entities and edges are merged so a node aggregates evidence corpus-wide.
LLM extraction replaces hand-built schemas and rule-based pipelines, trading setup cost for variability.
Indexing is token-heavy and prompt-sensitive; extraction quality bounds retrieval quality.

Community detection and summarization

Once the graph exists, GraphRAG partitions it into communities, groups of entities that are more densely connected to each other than to the rest of the graph. The Microsoft implementation uses the Leiden algorithm, run hierarchically via the graspologic library. Leiden is a community-detection method that improves on the older Louvain algorithm by guaranteeing that the communities it returns are internally well connected. Running it hierarchically produces nested levels: fine-grained communities at the bottom and progressively broader ones above.

For each community, the system asks a language model to write a report-style summary of the entities and relationships it contains. These community summaries are precomputed during indexing, not at query time. They are the mechanism that lets GraphRAG answer global questions: instead of trying to stuff an entire corpus into a context window, the system reasons over a tractable set of summaries that already abstract the corpus at a chosen level of granularity.

The hierarchy matters because it gives the query stage a granularity dial. A narrow question can be answered from low-level communities near the relevant entities, while a broad "what are the main themes" question can be answered from a handful of top-level community summaries. Community size is bounded by a maximum-cluster-size parameter, so no single community grows too large to summarize faithfully.

globalAnswer = reduce( map(qi, summary(C)) for C in communities(level=L) )

The map-reduce shape of a global GraphRAG query: each selected community summary yields a partial answer to question qi, and the partials are reduced into one final response.

Leiden community detection (run via graspologic) partitions the graph into well-connected clusters.
Hierarchical Leiden yields nested community levels, from fine-grained to corpus-wide.
An LLM precomputes a summary report for each community during indexing, not at query time.
A maximum-cluster-size cap keeps each community small enough to summarize faithfully.

Querying: local, global, and multi-hop reach

GraphRAG exposes more than one query method because different questions need different traversals. A local search answers questions about specific entities by gathering a neighborhood: the entity's own description, its relationships, the text units it came from, and nearby community context. This is close in spirit to ordinary RAG but enriched with graph structure, so it can follow a relationship edge to reach context a pure vector match would have ranked too low.

A global search answers corpus-wide questions by working over community summaries at a chosen hierarchy level. Each relevant community summary produces a partial answer, and those partial answers are aggregated into a final response. This map-reduce pattern over summaries is what lets GraphRAG handle "sensemaking" queries, questions whose answer is distributed across the whole dataset and therefore invisible to any single retrieved chunk. The Microsoft tooling also ships additional methods, including a DRIFT search that blends local and global signals and a basic vector method for comparison.

The practical payoff is multi-hop reach. When an answer requires connecting entity A to entity C through an intermediate B that the query never mentions, flat RAG often fails because no chunk is simultaneously similar to A and C. A graph can traverse the A to B to C path explicitly. The cost is real: GraphRAG's index is expensive to build and to keep fresh, so it pays off most on stable corpora and on question types where global comprehensiveness matters more than latency.

bash

# Microsoft GraphRAG: index-then-query flow (graphrag PyPI package)
pip install graphrag

# 1. Scaffold config + prompts in the project root
graphrag init --root ./ragtest

# put source text files in ./ragtest/input, set your model key in settings.yaml

# 2. Build the knowledge graph + community summaries (expensive, offline)
graphrag index --root ./ragtest

# 3a. Global query: corpus-wide sensemaking over community summaries
graphrag query --root ./ragtest --method global \
  "What are the top themes across these documents?"

# 3b. Local query: entity-centric neighborhood retrieval
graphrag query --root ./ragtest --method local \
  "How is entity X related to entity Y?"

The init then index then query lifecycle using Microsoft's open-source graphrag package; method selects local vs global traversal.

Local search gathers an entity's neighborhood, relationships, and source text units.
Global search runs map-reduce over community summaries for corpus-wide answers.
Multi-hop questions that connect entities through unmentioned intermediates favor graph traversal.
The tooling adds DRIFT (hybrid) and basic (vector) methods alongside local and global.

When GraphRAG helps, and when it does not

GraphRAG earns its complexity on questions that flat RAG handles badly: global sensemaking ("what are the main themes"), multi-hop reasoning across entities, and queries where comprehensiveness and diversity of the answer matter. The original Microsoft evaluation reported gains in comprehensiveness and diversity over conventional vector RAG on broad questions over datasets of roughly one million tokens. The strength comes from precomputed structure: the corpus is digested once into a graph and summaries, so the query stage reasons over abstractions rather than raw fragments.

The weaknesses are equally concrete. The index is costly in both tokens and time because every text unit is sent through an extraction model, and any change to the corpus requires re-indexing or an incremental update. Answer quality is capped by extraction quality, so noisy or ambiguous source text yields a noisy graph. For simple fact-lookup questions where one passage holds the answer, the extra machinery adds cost and latency without improving the result; plain vector retrieval, often paired with reranking, is the better fit there.

Because of this, GraphRAG is frequently deployed as one tool in a hybrid retrieval stack rather than a wholesale replacement for embeddings. A system might route narrow lookups to vector search and route broad, exploratory questions to a graph index, or keep both signals available to the same query planner. The decision hinges on corpus stability, question mix, and tolerance for indexing cost.

Best for global sensemaking, multi-hop reasoning, and answer comprehensiveness over a corpus.
Reported gains in comprehensiveness and diversity on roughly million-token datasets.
Weak fit for single-passage fact lookups, where vector search plus reranking is cheaper.
Often deployed inside a hybrid stack rather than replacing vector retrieval outright.

Key takeaways

GraphRAG grounds answers in an LLM-built knowledge graph of entities and relationships, not only flat vector chunks.
It is associated with Microsoft Research's 2024 paper "From Local to Global: A Graph RAG Approach to Query-Focused Summarization" (arXiv:2404.16130) and its open-source graphrag package.
Indexing extracts entities and relations with an LLM, then detects communities with the hierarchical Leiden algorithm and precomputes a summary for each.
Global queries map-reduce over community summaries for corpus-wide sensemaking; local queries gather an entity's neighborhood.
Its advantage is multi-hop reach and comprehensiveness on broad questions that flat top-k retrieval misses.
Its cost is expensive, prompt-sensitive indexing, so it suits stable corpora and is often paired with, not a replacement for, vector search.

Frequently asked questions

Regular RAG retrieves the text chunks most similar to a query from a vector store and feeds them to the model. GraphRAG first builds a knowledge graph of entities and relationships from the corpus and detects communities within it, then answers by traversing that graph or by summarizing communities. This lets it handle multi-hop and corpus-wide questions where no single chunk contains the answer.

Local search answers questions about specific entities by gathering their graph neighborhood: descriptions, relationships, and source text units. Global search answers broad, corpus-wide questions by running a map-reduce over precomputed community summaries, generating partial answers from each relevant community and combining them. Microsoft's tooling also ships DRIFT and basic methods alongside these two.

Leiden is a community-detection algorithm that groups densely connected entities into clusters and guarantees those clusters are internally well connected, improving on the older Louvain method. GraphRAG runs it hierarchically (via the graspologic library) to produce nested community levels, which gives the query stage a granularity dial from fine-grained neighborhoods up to corpus-wide themes.

The indexing phase is the costly part, because every text unit is sent through a language model to extract entities and relationships, and community summaries are generated up front. Microsoft's documentation warns that indexing can be expensive and recommends tuning extraction prompts first. Query-time cost is comparatively modest, so GraphRAG suits relatively stable corpora where the one-time index can be amortized.

Use GraphRAG for global sensemaking, multi-hop reasoning across entities, and questions where answer comprehensiveness matters more than latency. For simple lookups where a single passage holds the answer, plain vector retrieval, often with a reranking step, is cheaper and just as accurate. Many production systems run both and route the question to whichever fits.

Not necessarily. Microsoft's open-source graphrag package builds and stores the graph and community summaries as files via its own pipeline, without requiring a dedicated graph database. Some teams do load the resulting graph into a graph database for interactive traversal or to integrate with existing infrastructure, but that is an optional deployment choice rather than a requirement.

Put the idea into practice

MemX is an AI memory agent built on these ideas: store anything, skip the folders, and find it again by asking in plain English.

Try MemX Free