GraphRAG from Scratch — Where Knowledge Graphs Meet Retrieval

Rebuild your documents as a web of entities and relationships and you can answer questions ordinary RAG cannot reach — the ones about the corpus as a whole. Extraction, entity resolution, community summarization, and local/global search from first principles, ending with an honest account of when it's overkill.

From Local to Global: A Graph RAG Approach to Query-Focused Summarization

"So what is this pile of documents actually about?"

Someone hands you a box of company email. If you want to know which projects Tanaka is on, that's easy: search for "Tanaka," read the handful of hits. The answer is written down inside some individual message.

Now change the question: "What are the top five issues this group keeps arguing about?" That answer isn't in any single message. Read them one by one and you'll pick up fragments — "estimates come in late," "the spec keeps changing" — but how often each one recurs, and between whom, only emerges after you've read everything and tallied it in your head. Search is a tool for finding things that resemble your query. It is not a tool for surveying a set and summarizing it.

Call the first kind a local question and the second a global question. GraphRAG is machinery for answering the global kind.

What ordinary RAG can and can't cover

Ordinary RAG (retrieval-augmented generation) splits documents into chunks, turns each into an embedding vector, and pulls the top-k nearest to the question into the prompt (full pipeline in RAG Fundamentals and Design Patterns). That design carries an assumption it rarely states out loud: the answer sits, whole, inside one of those chunks.

The assumption breaks when the answer is scattered one sentence at a time across a hundred chunks. Retrieving five of them gives you a biased 5% sample — and the model will confidently declare "the three main issues are as follows" on the basis of it. Retrieval has failed, but the output doesn't look like failure. That is the characteristic accident of asking a global question of a RAG system.

Nor can you simply pass everything. You blow past the context limit, and where you don't, cost and latency go with it. Drag the query in the figure below: however you move it, what comes back is a few neighbors of the query, never a picture of the whole set.

The knowledge graph: rebuilding documents as dots and lines

GraphRAG's move is not to make retrieval smarter. It is to change the shape of the documents before you retrieve at all. The new shape is a knowledge graph, and it has exactly two ingredients:

• Nodes: the things a document talks about — people, organizations, products, concepts. These are entities
• Edges: relationships between them. "Tanaka —(works in)→ Engineering," "Engineering —(owns)→ Project X"

What this buys you first is aggregation. Tanaka may appear scattered across two hundred emails, but in the graph he is one node, and all two hundred relationships attach to it. The whole picture that previously required reading two hundred places is now available by looking at a single node.

The second thing is that edges are traversable. "Tanaka → Project X → Client Y" is a two-hop path that reconstructs a relationship written down nowhere as a continuous statement. Embedding similarity cannot reach this in principle: Tanaka's documents and Y's documents share no vocabulary, and only the hop through X connects them.

Step 1: extract entities and relationships

To pull triples out of prose, you use an LLM. Per chunk, the instruction is roughly this:

Extract the entities of the given types, and the relationships between them.
Entity types: [person, organization, product, event]
Output: (name, type, description) and (entity A, entity B, relationship, strength 1-10)

The load-bearing part is that you supply the entity types (entity_types). Leave it open and the model promotes generic nouns — "meeting," "issue," "response" — into nodes, and the graph turns to mud. A human deciding up front what the protagonists of this domain are is most of the eventual quality.

The standard implementation trick is gleaning: a single extraction pass always misses some fraction, so you ask the same chunk a few more times — "are there entities you haven't extracted yet?" — and add what comes back. A plain trade of recall against LLM calls. And because extraction is per-chunk, your chunk boundaries become your graph quality: a relationship stated in one sentence that got cut in half is an edge that never existed (Chunking Strategies).

Step 2: collapse the duplicates

The graph you get straight out of extraction is unusable, because "Tanaka Taro," "Tanaka," and "T. Tanaka" are three separate nodes. Aggregation — the whole point — hasn't happened.