Graph-R1: Agentic GraphRAG with RL for Multi-Turn Reasoning

Large language models (LLMs) have revolutionized natural language processing, yet their persistent tendency to generate inaccurate or fabricated information, often termed “hallucination,” remains a significant hurdle for applications requiring high factual accuracy. Retrieval-Augmented Generation (RAG) frameworks offer a partial solution by incorporating external knowledge, but traditional RAG systems often fall short. They typically rely on retrieving discrete text segments, which struggle to capture complex semantic relationships. While more advanced GraphRAG methods, which use structured knowledge graphs, address some of these limitations, they frequently incur high construction costs, lack flexibility in retrieval, and depend heavily on lengthy context windows and meticulously crafted prompts.

Addressing these challenges, a collaborative research effort from Nanyang Technological University, National University of Singapore, Beijing Institute of Computer Technology and Application, and Beijing Anzhen Hospital has unveiled Graph-R1. This innovative framework represents a significant leap forward, utilizing an agentic GraphRAG approach powered by end-to-end reinforcement learning to facilitate structured, multi-turn reasoning.

Graph-R1 introduces several core innovations that set it apart. First, it employs a lightweight method for constructing a knowledge hypergraph. Unlike simpler graphs, this hypergraph uses LLM-driven n-ary relation extraction to encode richer, more semantically grounded relationships between concepts. This approach boosts the system’s reasoning capabilities while maintaining remarkable efficiency. For instance, constructing this complex graph costs only $2.81 per 1,000 tokens and takes a mere 5.69 seconds, a notable improvement over GraphRAG ($3.35) and HyperGraphRAG ($4.14). Despite its efficiency, the resulting graphs are semantically rich, featuring over 120,000 nodes and nearly 100,000 edges.

Second, Graph-R1 features a sophisticated multi-turn agentic retrieval process. Rather than a single, static retrieval attempt, the system models knowledge retrieval as an iterative “think-retrieve-rethink-generate” loop. This dynamic interaction allows the AI agent to adaptively query and refine its knowledge path, exploring the hypergraph until it determines the most relevant information. This process intelligently fuses entity-based and hyperedge retrieval through a combined ranking mechanism, significantly enhancing the likelihood of pinpointing the most pertinent knowledge.

Finally, Graph-R1 optimizes its entire operation using end-to-end reinforcement learning, specifically through Group Relative Policy Optimization (GRPO). This unified training approach integrates rewards for adherence to output format, relevance of retrieved information, and overall answer correctness. By guiding the agents with this comprehensive reward mechanism, Graph-R1 develops generalizable reasoning strategies that are tightly aligned with both the underlying knowledge structure and the quality of the generated output. This means the system is rewarded not just for correct answers, but for arriving at them through structurally valid and logical reasoning trajectories.

Empirical evaluations underscore Graph-R1’s superior performance. Benchmarked across six standard question-answering datasets, including 2WikiMultiHopQA and HotpotQA, Graph-R1 achieved an average F1 score of 57.82 using the Qwen2.5-7B model. This figure substantially outperforms all previous baselines, demonstrating a wide margin of improvement over methods like NaiveGeneration (13.87), StandardRAG (15.89), GraphRAG (24.87), and HyperGraphRAG (29.40). The research also indicates that leveraging larger base models further amplifies these performance gains.

Ablation studies, which test the necessity of each component, confirmed that removing any of Graph-R1’s core modules—hypergraph construction, multi-turn reasoning, or reinforcement learning optimization—leads to a dramatic reduction in performance, validating the critical role of each innovation. Furthermore, Graph-R1’s retrieval process is not only more effective but also more concise and efficient. It achieves high F1 scores with moderate average content lengths of approximately 1,200 to 1,500 tokens per exchange, supporting an average of 2.3 to 2.5 interaction turns for stable and accurate knowledge extraction. In terms of generation cost, Graph-R1 maintains minimal overhead, boasting a response time of 7.0 seconds per query and effectively zero cost per query, significantly outperforming competitors like HyperGraphRAG, which incurs $8.76 per query and takes 9.6 seconds.

When assessed across seven dimensions of generation quality—including comprehensiveness, correctness, relevance, and logical coherence—Graph-R1 consistently outperformed all other RL-based and graph-based baselines, achieving top scores in correctness (86.9), relevance (95.2), and coherence (88.5). Its generalizability was also robustly demonstrated through cross-validation on out-of-distribution settings, where it maintained strong performance, often exceeding 85% of its in-distribution ratios, highlighting its adaptability across diverse datasets.

The theoretical underpinnings of Graph-R1 provide further insights into its effectiveness. Information-theoretic analyses suggest that its graph-structured knowledge offers higher information density per retrieval and faster convergence to correct answers compared to traditional chunk-based methods. The multi-turn interaction empowers the agent to achieve greater retrieval efficiency by dynamically focusing on high-impact regions of the graph. Finally, the end-to-end reinforcement learning optimization effectively bridges the gap between structured graph evidence and natural language generation, thereby reducing output entropy and error rates.

By integrating hypergraph-based knowledge representation, agentic multi-turn reasoning, and end-to-end reinforcement learning, Graph-R1 delivers unprecedented gains in factual question-answering performance, retrieval efficiency, and generation quality. This framework charts a promising path for the development of next-generation agentic and knowledge-driven LLM systems, particularly in complex, knowledge-intensive domains such as healthcare, legal, and enterprise knowledge automation, where factual accuracy and transparent reasoning are paramount.