GraphRAG (Knowledge Graph Retrieval-Augmented Generation)

What is GraphRAG?

GraphRAG, or Knowledge Graph Retrieval-Augmented Generation, is a technique that combines a knowledge graph with vector retrieval so large language models can reason over private data more completely than standard retrieval-augmented generation allows. Standard RAG systems typically rely on vector databases to retrieve text chunks based on semantic similarity. While effective for simple point-in-time factual lookups, standard RAG struggles with complex questions that require synthesizing information across multiple documents or following relationships between entities.

GraphRAG addresses this by introducing a structural layer over the raw text. It uses LLMs during the indexing phase to process documents, identify key entities (like people, organizations, concepts, and locations), and map the relationships between them. This information is stored in a knowledge graph, a network of interconnected data points. When a user asks a question, the system can traverse this graph, following the edges between nodes to gather context that might be spread across numerous disparate documents. Microsoft Research's GraphRAG project popularized the hierarchical community-summary approach described below.

By structuring unstructured data into a graph, GraphRAG enables LLMs to answer "global" questions that require summarizing entire datasets (e.g., "What are the main themes in this document collection?") and "multi-hop" questions that require following a chain of evidence (e.g., "How does Company A's acquisition of Company B affect Person C?"). It bridges the gap between the structured world of databases and the unstructured world of text, offering a more holistic, interconnected view of information.

Vector RAG vs. GraphRAG

Understanding when to use GraphRAG requires comparing it to traditional vector-based RAG. They are not mutually exclusive; in fact, the most advanced systems often combine both within an agentic RAG pipeline that routes each query to the right retrieval method. Their core strengths differ significantly.

Traditional Vector RAG excels at speed and cost-efficiency. If a user asks "What is the capital of France?" a vector search quickly identifies the chunk containing the answer. However, if asked, "What are the overarching themes in our customer feedback regarding our new product launch?" Vector RAG will likely just retrieve a few random feedback chunks, missing the broader picture. GraphRAG, through its community summaries, can easily synthesize those overarching themes.

How GraphRAG Works: Entity Extraction, Community Clustering, and Summarization

GraphRAG's indexing pipeline transforms raw text into a structured, queryable knowledge graph in three phases:

1. Entity and Relationship Extraction

The first step is parsing the source documents. Unlike standard RAG, which merely chunks and embeds text, GraphRAG employs an LLM to actively read and interpret the chunks. The LLM is prompted to extract specific entities, which could be anything relevant to the domain, such as companies, individuals, diseases, or legal concepts.

The LLM also extracts the relationships between these entities, identifying, for example, that "Microsoft" "invested in" "OpenAI." These entities become the nodes of the knowledge graph, and the relationships become the edges. This phase is computationally intensive, as the LLM must process the entire corpus to build a comprehensive map of the data.

2. Knowledge Graph Construction and Community Clustering

Once the entities and relationships are extracted, they are assembled into a formal knowledge graph. However, real-world data is messy, and graphs can quickly become overwhelmingly complex. To make the graph useful for retrieval, GraphRAG employs community detection algorithms (like the Louvain method or Leiden algorithm).

These algorithms analyze the structure of the graph to identify clusters of nodes that are densely connected to each other but sparsely connected to the rest of the graph. These clusters, or "communities," represent thematic groupings within the data. For example, in a corpus of news articles, one community might form around "technology companies," while another forms around "political figures." The graph is often clustered hierarchically, creating communities at various levels of granularity (e.g., a broad "technology" community containing smaller "artificial intelligence" and "hardware" sub-communities).

3. Community Summarization

The final indexing step is what makes GraphRAG effective for global queries. The system takes each identified community and uses an LLM to generate a summary of it. The LLM is provided with the entities, relationships, and claims within that specific community and asked to synthesize a coherent overview.

This process is repeated for every community at every level of the hierarchy. The result is a pre-computed set of summaries that describe the entire dataset from different thematic angles and at different levels of detail. When a user asks a high-level question, the system doesn't need to read thousands of documents; it can simply retrieve and combine these pre-computed community summaries.

Query Modes: Local Search vs. Global Search

GraphRAG systems typically offer different querying modes optimized for the type of question being asked. The two primary modes are Local Search and Global Search.

Local Search (Entity-Centric Queries)

Local search is designed for questions focused on specific entities or concepts. It's the graph equivalent of a targeted investigation. When a user asks a question like, "What are all the known associations of Person X?" the system operates as follows:

1. Entity Resolution: The system identifies the key entities mentioned in the user's query (e.g., "Person X").
2. Neighborhood Extraction: It locates those entities within the knowledge graph and extracts their immediate "neighborhood," the nodes directly connected to them and the relationships (edges) between them.
3. Context Assembly: The system gathers the textual data associated with those nodes and edges.
4. Generation: An LLM uses this highly relevant, localized graph context to generate a comprehensive answer.

Local search is effective for multi-hop reasoning. If the system knows A is connected to B, and B is connected to C, it can answer questions about the relationship between A and C, something standard vector RAG struggles with.

Global Search (Corpus-Level Queries)

Global search is designed for questions that require an understanding of the entire dataset. These are questions that can't be answered by looking at any single document or entity. For example, "What are the major geopolitical trends mentioned in these reports?"

Instead of searching for specific entities, global search leverages the hierarchical community summaries generated during the indexing phase.

1. Summary Retrieval: The system retrieves the pre-computed summaries for all communities at a specified level of the hierarchy (e.g., the top-level, broadest communities).
2. Map-Reduce Synthesis: Because these summaries might still be too large to fit into a single LLM context window, GraphRAG employs a map-reduce strategy. It chunks the summaries, asks an LLM to generate partial answers based on each chunk (the "map" step), and then asks the LLM to synthesize those partial answers into a final, comprehensive response (the "reduce" step).

This approach allows GraphRAG to synthesize information across massive corpora, providing insights that would be impossible to derive using standard retrieval methods.

After Microsoft GraphRAG: DRIFT Search and LazyGraphRAG

Microsoft's original GraphRAG (paper published April 2024) proved the community-summary approach worked, but its indexing bill made teams hesitate. Two follow-ups from the same group changed the cost calculus.

DRIFT search (Dynamic Reasoning and Inference with Flexible Traversal, added to the GraphRAG library in late 2024) blends local and global search in a single query mode. It starts from community summaries to get a broad view of the question, then drills into specific entities, so one query can answer questions that previously required choosing the right mode up front.

LazyGraphRAG (announced November 2024, later merged into the GraphRAG library) removes the expensive part entirely. Instead of using an LLM to extract entities and relationships at indexing time, it builds a lightweight concept graph from NLP noun-phrase extraction and co-occurrence, then defers all LLM work to query time, where an iterative best-first search summarizes only the subgraph the query touches. Microsoft's benchmarks on 5,590 AP news articles put its indexing cost at the same level as vector RAG, about 0.1% of full GraphRAG's, while matching GraphRAG global search quality at more than 700 times lower query cost.

Outside Microsoft, lighter-weight open-source variants such as LightRAG (HKU, October 2024) took a similar direction: simpler graph construction, incremental updates, and dual-level retrieval that mixes entity lookups with broader theme retrieval. The pattern across all of these is the same. The field kept the graph structure and dropped the assumption that every node and edge must be extracted by an expensive LLM pass before the first query arrives.

For a team evaluating GraphRAG in 2026, the practical default is to start with a lazy or lightweight variant and only pay for full LLM-extracted graphs when the domain demands typed, auditable relationships (compliance, biomedical, intelligence work).

Production Considerations & Indexing Costs

While GraphRAG offers significant advantages, deploying it in production requires careful planning, primarily due to the increased costs and complexity associated with the indexing phase.

LLM Extraction Costs

The most significant barrier to adopting GraphRAG is the cost of indexing. Unlike vector RAG, which relies on relatively inexpensive word embedding models, GraphRAG requires using powerful (and costly) LLMs to extract entities, relationships, and claims from every chunk of text in the corpus.

If you have a million documents, you must make millions of LLM calls just to build the graph. This can quickly become prohibitively expensive, especially if you are using frontier models like GPT-5 or Claude Opus 4.5. This is exactly the cost that LazyGraphRAG was built to avoid, and why lazy indexing is now the usual starting point.

To manage these costs, organizations often employ strategies such as:

• Model Routing: Using smaller, cheaper models (like Qwen3 or fine-tuned BERT models) for the initial entity extraction phase, and reserving larger models only for complex summarization or generation tasks.
• Targeted Extraction: Instead of extracting everything, constraining the LLM to only look for specific entity types relevant to the business use case (e.g., only extracting "Organizations" and "Locations").
• Incremental Indexing: Only updating the graph with new information, rather than rebuilding it from scratch when the corpus changes.

Graph Database Choices

To implement GraphRAG effectively, you need a database capable of storing and querying both vector embeddings and graph structures. Several architectural patterns are emerging:

• Dedicated Graph Databases: Systems like Neo4j are explicitly designed for graph data. They offer powerful query languages (like Cypher) for complex graph traversal. Many modern graph databases are now adding vector search capabilities to support hybrid GraphRAG workflows.
• Multi-Model Databases: Databases like ArangoDB support both graph and document/vector structures natively, simplifying the architecture.
• Vector Databases with Graph Capabilities: Some vector databases are beginning to add features to support graph-like relationships between vectors, though they may lack the deep traversal capabilities of dedicated graph databases.
• Relational Databases (with Extensions): In some cases, traditional relational databases (like PostgreSQL with the pgvector extension and recursive CTEs) can be adapted to handle basic GraphRAG workloads, though scaling complex traversals can be challenging.

The right choice depends on the scale of the data, the complexity of the relationships, and the existing infrastructure of the organization.

Frequently Asked Questions

Does GraphRAG replace standard Vector RAG? No. GraphRAG and Vector RAG are complementary. Vector RAG is generally faster, cheaper, and better for simple, fact-based queries. GraphRAG excels at complex, multi-hop reasoning and global summarization. Most production systems use a hybrid approach, routing queries to the appropriate retrieval method based on the user's intent.

Should I use full GraphRAG or LazyGraphRAG? Start lazy. LazyGraphRAG indexes at roughly vector-RAG cost and matched full GraphRAG's global-search quality in Microsoft's own evaluation, so the burden of proof sits on the expensive option. Full LLM-extracted graphs earn their cost when you need the graph itself as an artifact: typed relationships you can query with Cypher, audit trails from node to source text, or entity data that feeds systems beyond the RAG pipeline.

Is GraphRAG only useful for massive datasets? No. GraphRAG is built for synthesizing large corpora, but it also helps on smaller, dense datasets where the relationships between entities are crucial. For example, analyzing a dense legal contract or a complex medical record can benefit significantly from graph-based extraction, even if the total word count is relatively low.

How do you handle graph updates when the underlying data changes? Updating a knowledge graph is more complex than updating a vector database. If a document is deleted or modified, you must remove its vector embeddings and identify and remove or update the entities, relationships, and community summaries derived from that document. This requires robust tracking of data provenance (knowing exactly which source text produced which graph node).

What happens if the LLM hallucinates relationships during the extraction phase? This is a significant risk. If the LLM incorrectly identifies a relationship (e.g., stating that Company A acquired Company B when they merely partnered), that error becomes baked into the knowledge graph and can degrade the accuracy of future answers. Mitigating this requires careful prompt engineering, using highly capable models for extraction, and implementing human-in-the-loop verification or automated validation steps where possible.