aiqbench analysis
This research introduces SAFE, a framework that uses an LLM as a verifier to enforce evidence-grounded reasoning during multi-hop question answering tasks. By decomposing reasoning into atomic units validated against Knowledge Graph triples, the method prevents error propagation and significantly boosts accuracy on standard benchmarks. The approach represents a critical shift from post-hoc evaluation to stepwise verification, addressing the issue of spurious correctness in current LLM capabilities.
SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning
arXiv:2604.01993v3 Announce Type: replace-cross Abstract: Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning. We propose SAFE, an LLM-as-verifier framework for evidence-grounded multi-hop QA. Rather than judging only the final answer after generation, SAFE verifies reasoning during generation by checking intermediate steps against the provided passages and previous reasoning trajectory. To make this process checkable, SAFE decomposes reasoning into atomic, evidence-grounded units represented with Knowledge Graph (KG) triples. At train-time, SAFE verifies benchmark supervision under KG-grounded constraints and constructs reliable verifier training data. At inference-time, an external verifier checks each generated step, identifies invalid reasoning, and provides correction feedback before errors propagate. Across three multi-hop QA benchmarks, SAFE improves accuracy by 8.8 pp on average. These results show that evidence-grounded multi-hop QA benefits from shifting LLM-based evaluation from post-hoc answer judgment to stepwise reasoning verification.
Abstract:Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning. We propose SAFE, an LLM-as-verifier framework for evidence-grounded multi-hop QA. Rather than judging only the final answer after generation, SAFE verifies reasoning during generation by checking intermediate steps against the provided passages and previous reasoning trajectory. To make this process checkable, SAFE decomposes reasoning into atomic, evidence-grounded units represented with Knowledge Graph (KG) triples. At train-time, SAFE verifies benchmark supervision under KG-grounded constraints and constructs reliable verifier training data. At inference-time, an external verifier checks each generated step, identifies invalid reasoning, and provides correction feedback before errors propagate. Across three multi-hop QA benchmarks, SAFE improves accuracy by 8.8 pp on average. These results show that evidence-grounded multi-hop QA benefits from shifting LLM-based evaluation from post-hoc answer judgment to stepwise reasoning verification.
Accepted to EMNLP 2026 (Main Conference). Project Page: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2604.01993 [cs.CL]
(or arXiv:2604.01993v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2604.01993 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning
arXiv · Jun 10, 2026
Models Can't Remember Their Training. Neither Can You.
LessWrong · Jul 19, 2026
News
Quantum Computing Report · Jul 25, 2026
Building an Enterprise AI Customer Support Platform with Claude Fable 5.1 and Claude Code
Analytics Vidhya · Oct 5, 2026