aiqbench analysis
This research identifies data integrity as the critical bottleneck in Graph-RAG systems, revealing that LLMs often mask storage deterioration by compensating with internal memory rather than fixing retrieval paths. By decoupling error attribution into reasoning, knowledge graph defects, and query generation components, the study provides a quantitative framework for auditing cloud-native information fusion systems where structural data flaws drastically reduce accuracy.
Decoupling Error Attribution in Cloud-Native Graph-RAG: A Data Integrity Diagnostic Framework
arXiv:2609.13324v1 Announce Type: cross Abstract: Graph-RAG systems often assume pristine data quality, overlooking the severe impact of perturbations in cloud-native databases. This paper proposes a three-layer decoupled diagnostic framework to orthogonally attribute system errors to reasoning loss, Knowledge Graph (KG) defects, and Cypher generation errors. Evaluated on a spatio-temporal ecological KG of the Southeastern Tibet region with eight defect types, results reveal that data integrity, rather than algorithmic reasoning, is the dominant performance bottleneck, with structural defects degrading system accuracy from 0.93 to 0.39. Crucially, we observe a masking-like phenomenon termed the Parametric Knowledge Masking Effect (PKME), suggesting LLMs compensate for broken retrieval paths using internal memory. This shrinks apparent query generation errors by over 70 percent, obscuring actual storage deterioration and increasing the risk of false negatives for automated monitoring. This work provides a quantitative foundation for auditing and optimizing data integrity in cloud-based information fusion systems.
Abstract:Graph-RAG systems often assume pristine data quality, overlooking the severe impact of perturbations in cloud-native databases. This paper proposes a three-layer decoupled diagnostic framework to orthogonally attribute system errors to reasoning loss, Knowledge Graph (KG) defects, and Cypher generation errors. Evaluated on a spatio-temporal ecological KG of the Southeastern Tibet region with eight defect types, results reveal that data integrity, rather than algorithmic reasoning, is the dominant performance bottleneck, with structural defects degrading system accuracy from 0.93 to 0.39. Crucially, we observe a masking-like phenomenon termed the Parametric Knowledge Masking Effect (PKME), suggesting LLMs compensate for broken retrieval paths using internal memory. This shrinks apparent query generation errors by over 70 percent, obscuring actual storage deterioration and increasing the risk of false negatives for automated monitoring. This work provides a quantitative foundation for auditing and optimizing data integrity in cloud-based information fusion systems.
Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
arXiv:2609.13324 [cs.IR]
(or arXiv:2609.13324v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2609.13324 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 240 words
