[3P1-OS-46a] OS-46

[3P1-OS-46a-02] Comparative Analysis of GraphRAG Based on Document Chunks and GraphRAG Based on Community Selection of Knowledge Graph

〇Fumikatsu Anaguchi^1,2, Sudesna Chakraborty¹, Takeshi Morita^1,2, Shusaku Egami², Takanori Ugai^2,3, Ken Fukuda² (1. Aoyama Gakuin University, 2. National Institute of Advanced Industrial Science and Technology (AIST), 3. Fujitsu Limited)

Keywords:Large Language Model, Retrieval-Augmented Generation, GraphRAG, Knowledge Graph, Document Search

Retrieval-Augmented Generation (RAG) has gained attention as an approach for Large Language Models (LLMs) to utilize external knowledge. In RAG, various extension methods have been explored for each module. In particular, retrieval modules have been extensively studied, with a primary focus on similarity retrieval methods using embedding vectors. Additionally, GraphRAG, which leverages knowledge graphs, is expected to structure information and identify relationships between entities. However, the structural differences and construction methodologies of knowledge graphs in GraphRAG remain unclear. In this study, we compare two GraphRAG approaches: one based on semantic chunking and the other based on community-based knowledge graph classification. Using an information-seeking QA dataset derived from academic publications, we analyze the differences in the knowledge graphs generated by these methods from the perspectives of domain knowledge representation and contextual understanding. Additionally, through a comparative evaluation of QA task performance, we discuss the existing challenges and potential improvements in GraphRAG methodologies.