analiza aiqbench
TriQua addresses the granularity-context trade-off in LLM fact verification by introducing hyperrelational facts to preserve essential context without sacrificing atomicity. The framework demonstrates superior decomposition quality and alignment with human annotations compared to existing methods, offering a robust approach for evidence-based evaluation. This advancement directly enhances the reliability of large language models through fine-grained error detection within structured factual units.
TriQua: Reconciling Granularity and Context in Factuality Evaluation
arXiv:2608.05228v1 Announce Type: cross Abstract: The "decompose-then-verify" paradigm for LLM factuality evaluation faces a fundamental trade-off: atomic facts, i.e., one sentence conveying one unit of information, often omit essential context, while broader statements lack the granularity needed for precise assessment. To address this, we introduce TriQua, a framework that flexibly models facts based on their complexity. Simple claims are extracted as standard triples, while complex claims are represented as hyperrelational facts by attaching auxiliary contextual qualifiers. This adaptive structure preserves the necessary context for accurate retrieval and verification without sacrificing atomicity. Furthermore, TriQua's verification process directly annotates concrete errors within specific triples and qualifiers, providing fine-grained explainability for error detection. Alongside the framework, we propose TriQuaScore to quantify the factuality of these structured fact units. Empirical evaluations show that TriQuaScore strongly aligns with human annotated factuality scores, TriQua achieves robust decomposition quality, and outperforms existing decomposition-based frameworks in evidence-based fact verification.
Abstract:The "decompose-then-verify" paradigm for LLM factuality evaluation faces a fundamental trade-off: atomic facts, i.e., one sentence conveying one unit of information, often omit essential context, while broader statements lack the granularity needed for precise assessment. To address this, we introduce TriQua, a framework that flexibly models facts based on their complexity. Simple claims are extracted as standard triples, while complex claims are represented as hyperrelational facts by attaching auxiliary contextual qualifiers. This adaptive structure preserves the necessary context for accurate retrieval and verification without sacrificing atomicity. Furthermore, TriQua's verification process directly annotates concrete errors within specific triples and qualifiers, providing fine-grained explainability for error detection. Alongside the framework, we propose TriQuaScore to quantify the factuality of these structured fact units. Empirical evaluations show that TriQuaScore strongly aligns with human annotated factuality scores, TriQua achieves robust decomposition quality, and outperforms existing decomposition-based frameworks in evidence-based fact verification.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2608.05228 [cs.AI]
(or arXiv:2608.05228v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.05228 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 244 cuvinte
Subiecte conexe
What We Learned Trying to Catch AI Liars: An Aletheia's Quest Retrospective
EleutherAI · 25 aug. 2026
Presentation: Building Reusable Evaluation Frameworks for Agentic AI Products
InfoQ · 5 oct. 2026
ACE-GraphRAG: Agentic Context Engineering for Hierarchical GraphRAG
arXiv · 4 aug. 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · 5 oct. 2026