aiqbench analysis
This paper reframes the persistent issue of LLM hallucinations as a fundamental problem of inaccurate internal world modeling rather than mere generation errors. By proposing a unified definition that distinguishes true factual failures from planning or reward-based issues, it establishes a clearer framework for benchmarking and mitigating these risks in frontier models. The introduction of the HalluWorld benchmark provides researchers with standardized tools to stress-test model reliability against fully specified reference world models.
A Unified Definition of Hallucination: It's The World Model, Stupid!
arXiv:2512.21577v3 Announce Type: replace Abstract: Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review existing definitions of hallucination and fold them into a single, unified definition wherein prior definitions are subsumed. We argue that hallucination can be unified by defining it as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base OR producing a summary which contradicts the source. By varying the reference world model and conflict policy, our framework unifies prior definitions. We argue that this unified view is useful because it forces evaluations to clarify their assumed reference "world", distinguishes true hallucinations from planning or reward errors, and provides a common language for comparison across benchmarks and discussion of mitigation strategies. Building on this definition, we also connect our framework to HalluWorld, a complementary benchmark that instantiates fully specified reference world models for stress-testing model hallucinations.
Abstract:Despite numerous attempts at mitigation since the inception of language models, hallucinations remain a persistent problem even in today's frontier LLMs. Why is this? We review existing definitions of hallucination and fold them into a single, unified definition wherein prior definitions are subsumed. We argue that hallucination can be unified by defining it as simply inaccurate (internal) world modeling, in a form where it is observable to the user. For example, stating a fact which contradicts a knowledge base OR producing a summary which contradicts the source. By varying the reference world model and conflict policy, our framework unifies prior definitions. We argue that this unified view is useful because it forces evaluations to clarify their assumed reference "world", distinguishes true hallucinations from planning or reward errors, and provides a common language for comparison across benchmarks and discussion of mitigation strategies. Building on this definition, we also connect our framework to HalluWorld, a complementary benchmark that instantiates fully specified reference world models for stress-testing model hallucinations.
ICML 2026. HalluWorld benchmark at this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2512.21577 [cs.CL]
(or arXiv:2512.21577v3 [cs.CL] for this version)
From: Steven Y. Feng [view email] [v1] Thu, 25 Dec 2025 08:42:18 UTC (1,561 KB) [v2] Tue, 3 Feb 2026 06:09:18 UTC (8,105 KB) [v3] Fri, 12 Jun 2026 23:25:06 UTC (8,109 KB)
Full text under the publisher's open licence · 228 words
Related coverage
What Hallucinations Have We Measured? Semantic Illusion, Confabulation, and Expressed Uncertainty
arXiv · Oct 30, 2026
Do Large Language Models Hallucinate Electric Fata Morganas?
arXiv · Aug 21, 2026
BEACON: Behavior-Anchored Cross-Source Knowledge Graph Construction for Cyber Threat Intelligence
arXiv · Aug 31, 2026
Fence: Specialized SLM Guardrails for LLM Applications
arXiv · Jul 23, 2026