Enhancing Multi-Hop Search with Graph RAG: A Knowledge Graph Approach

Graph RAG: Knowledge Graph Extraction for Multi-Hop Search

We often encounter this scenario: a standard vector RAG retrieves relevant chunks perfectly but cannot answer "How are company X and contract Y related?" It lacks the ability to understand relationships between entities and traverse a graph of connections. Graph RAG solves this by adding knowledge graph structure to embeddings. The system traverses the graph from a found entity through relationships to related concepts. This delivers better answers for complex multi-hop questions. Time savings on search reach up to 70%. Implementation pays off within 3–6 months. Manual analysis costs drop by up to 80%. For example, a legal department with 10,000 contracts might spend $50,000 annually on manual analysis; Graph RAG can reduce that to $10,000.

Why Standard RAG Falls Short on Multi-Hop Queries

Our case: a legal department with thousands of contracts over a long period. Standard RAG could not answer "Which suppliers participated in tenders where the winner was later declared bankrupt?" This required traversing the chain "tender → winner → bankruptcy." Graph RAG raised accuracy on such questions from 12% to 71%. The graph contained 45,000 entities and 180,000 relationships, built on Neo4j.

How Graph RAG Achieves Higher Accuracy: Graph Traversal

The key difference is graph traversal. When a user asks "Which contracts will be affected by the change of CEO in company X?", standard RAG finds chunks mentioning "CEO change X" but cannot infer that the CEO manages certain contracts through departments. Graph RAG follows the relationships: CEO → department → contract, obtaining full context. Our measurements on corporate documentation showed multi-hop accuracy rising from 12% to 71% and global summarization from 34% to 82%. For simple facts, Graph RAG matches standard RAG within 3%. Graph RAG outperforms standard RAG by 6x on multi-hop queries.

Architecture of Microsoft GraphRAG

The Microsoft GraphRAG architecture (Microsoft GraphRAG](https://microsoft.github.io/graphrag/) is the most influential implementation. The process includes:

1. LLM (GPT-4o) extracts entities and relationships from documents.
2. The built knowledge graph is stored in NetworkX or Neo4j.
3. The Leiden algorithm discovers hierarchical communities. A community report is generated for each.
4. Two search modes: Local — combines vector search with graph traversal from found entities; Global — summarizes community reports for global questions.

from openai import OpenAI
import json

client = OpenAI()

ENTITY_EXTRACTION_PROMPT = """Extract entities and relationships from the following text.
Return JSON:
{
  "entities": [
    {"id": "1", "name": "...", "type": "PERSON|ORG|CONTRACT|REGULATION|CONCEPT", "description": "..."}
  ],
  "relationships": [
    {"source": "id1", "target": "id2", "relation": "SIGNED|MANAGES|REFERS_TO|PART_OF", "description": "..."}
  ]
}

Text:
{text}"""

def extract_graph_elements(text: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": ENTITY_EXTRACTION_PROMPT.format(text=text)}],
        response_format={"type": "json_object"},
        temperature=0,
    )
    return json.loads(response.choices[0].message.content)

Building a Knowledge Graph with NetworkX

import networkx as nx
from typing import List

class KnowledgeGraph:
    def __init__(self):
        self.graph = nx.DiGraph()
        self.entity_embeddings = {}

    def add_elements(self, elements: dict, source_doc: str):
        for entity in elements["entities"]:
            self.graph.add_node(
                entity["id"],
                name=entity["name"],
                type=entity["type"],
                description=entity["description"],
                source=source_doc,
            )
        for rel in elements["relationships"]:
            self.graph.add_edge(
                rel["source"],
                rel["target"],
                relation=rel["relation"],
                description=rel["description"],
            )

    def get_subgraph(self, entity_id: str, depth: int = 2) -> nx.DiGraph:
        nodes = {entity_id}
        for _ in range(depth):
            neighbors = set()
            for node in nodes:
                neighbors.update(self.graph.predecessors(node))
                neighbors.update(self.graph.successors(node))
            nodes.update(neighbors)
        return self.graph.subgraph(nodes)

    def serialize_subgraph(self, subgraph: nx.DiGraph) -> str:
        lines = []
        for node in subgraph.nodes(data=True):
            lines.append(f"Entity: {node[1].get('name')} ({node[1].get('type')})")
            lines.append(f"  Description: {node[1].get('description', '')}")
        for edge in subgraph.edges(data=True):
            source_name = subgraph.nodes[edge[0]].get("name", edge[0])
            target_name = subgraph.nodes[edge[1]].get("name", edge[1])
            lines.append(f"Relation: {source_name} -> {target_name} ({edge[2].get('relation')})")
            lines.append(f"  {edge[2].get('description', '')}")
        return "\n".join(lines)

Local Search: Combining Graph and Vector Context

from langchain_openai import OpenAIEmbeddings
import numpy as np

class GraphRAGRetriever:
    def __init__(self, knowledge_graph: KnowledgeGraph, vectorstore, embeddings):
        self.kg = knowledge_graph
        self.vectorstore = vectorstore
        self.embeddings = embeddings

    def local_search(self, query: str, top_k: int = 5) -> str:
        vector_docs = self.vectorstore.similarity_search(query, k=top_k)
        mentioned_entities = self._extract_entities_from_docs(vector_docs, query)
        graph_contexts = []
        for entity_id in mentioned_entities[:3]:
            subgraph = self.kg.get_subgraph(entity_id, depth=2)
            graph_context = self.kg.serialize_subgraph(subgraph)
            graph_contexts.append(graph_context)
        vector_context = "\n\n".join([d.page_content for d in vector_docs])
        graph_context = "\n\n".join(graph_contexts)
        return f"## Text Context\n{vector_context}\n\n## Knowledge Graph Context\n{graph_context}"

Tools for Graph RAG

• Microsoft GraphRAG library: pip install graphrag — full implementation from Microsoft
• Neo4j + LangChain: Neo4jGraph + GraphCypherQAChain for Cypher queries
• LlamaIndex + Knowledge Graph: KnowledgeGraphIndex
• NetworkX: lightweight in-memory graph in Python

What’s Included in Our Work

• Design of the knowledge graph schema (entities, relationships, types)
• Implementation of the extraction pipeline using GPT-4o or Claude 3.5
• Graph construction with Neo4j or NetworkX
• Configuration of both Local and Global search modes
• Integration with your existing RAG system (LangChain, LlamaIndex)
• Testing on your data: accuracy measurements (precision/recall) and p99 latency
• Documentation and team training
• Dedicated support and maintenance

Our team has over 5 years of experience in NLP and has delivered 20+ production RAG systems. We guarantee high accuracy and reliable performance. Our certified engineers will ensure a smooth deployment. Contact us for a free consultation to assess your project.

Project Timeline

Pricing is individual — depends on document volume, required accuracy, and graph schema complexity. Request a detailed estimate.