aiqbench analysis
The paper examines how the labels used to evaluate LLM hallucination detectors are built. Using BLEURT similarity as a stand-in for factual error produces labels that disagree heavily with human judgment, with misclassification rates near 47% for BLEURT variants versus 12.86% for the best judge tested. For practitioners, the takeaway is that reported hallucination-detection scores depend heavily on the supervision method, so benchmark comparisons need scrutiny of how ground truth was produced.
When Ground Truth Is Not Ground Truth: Automatic Labeling for Hallucination Detection
arXiv:2611.04697v1 Announce Type: cross Abstract: Hallucinations in Large Language Models (LLMs) remain a major challenge, as they can produce misleading, factually incorrect, or unverifiable information, thereby undermining their trustworthiness and reliability. However, reliable ground-truth labels remain a key challenge in evaluating hallucination detectors. Recent studies often construct labels automatically by comparing model generations with reference answers using similarity metrics such as BLEURT. This implicitly treats BLEURT similarity as a proxy for hallucination, despite its English-centric supervision, unbounded score range, and design as a similarity metric rather than a factuality measure, making threshold-based ground-truth construction potentially unreliable. Through comprehensive experiments using 15 automatic supervision judges, seven hallucination detection baselines, two generator models, two QA datasets, and five languages, we demonstrate that detector performance varies substantially with the choice of supervision judge. Human validation reveals substantial differences in labeling reliability, with QGen-9B achieving the lowest overall misclassification rate (12.86%), while BLEURT and BLEURT-n exhibit rates of 47.23% and 47.32%, respectively. Our findings highlight the limitations of automatic similarity-based labeling and the need for more reliable judging methods that better capture semantic and factual correctness.
Abstract:Hallucinations in Large Language Models (LLMs) remain a major challenge, as they can produce misleading, factually incorrect, or unverifiable information, thereby undermining their trustworthiness and reliability. However, reliable ground-truth labels remain a key challenge in evaluating hallucination detectors. Recent studies often construct labels automatically by comparing model generations with reference answers using similarity metrics such as BLEURT. This implicitly treats BLEURT similarity as a proxy for hallucination, despite its English-centric supervision, unbounded score range, and design as a similarity metric rather than a factuality measure, making threshold-based ground-truth construction potentially unreliable. Through comprehensive experiments using 15 automatic supervision judges, seven hallucination detection baselines, two generator models, two QA datasets, and five languages, we demonstrate that detector performance varies substantially with the choice of supervision judge. Human validation reveals substantial differences in labeling reliability, with QGen-9B achieving the lowest overall misclassification rate (12.86%), while BLEURT and BLEURT-n exhibit rates of 47.23% and 47.32%, respectively. Our findings highlight the limitations of automatic similarity-based labeling and the need for more reliable judging methods that better capture semantic and factual correctness.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2611.04697 [cs.CL]
(or arXiv:2611.04697v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.04697 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
Who Checks the Citations? Benchmarking Legal Hallucination Detection
arXiv · Aug 7, 2026
The Hallucination Snowball: Modeling Error Propagation as State Transitions in Multi-Agent LLM Pipelines
arXiv · Aug 18, 2026
QUACK: Questioning, Understanding, and Auditing Communicated Knowledge in Multimodal Social Deduction Agents
arXiv · Sep 1, 2026
Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations
arXiv · Aug 5, 2026