analiza aiqbench
This study moves beyond surface-level taxonomy to pinpoint specific computational layers—self-attention, pretraining objectives, and autoregressive commitment—as the structural origins of LLM hallucinations. By demonstrating that substituting correct continuations significantly reduces downstream errors, the research provides a rigorous framework for diagnosing model failures rather than merely treating symptoms. These findings offer critical insights for developers aiming to architect more robust models by addressing the precise mechanisms where data pathologies and algorithmic biases intersect.
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
arXiv:2606.07537v2 Announce Type: replace Abstract: Large language models produce fluent, confident, factually wrong output. Existing taxonomies classify these failures by output type -- intrinsic versus extrinsic, faithfulness versus factuality -- but say nothing about which computational component produced a given failure. We ask what would be required to attribute an individual hallucination to a specific component of the decoder-only stack. We treat three components -- self-attention's associative retrieval, the maximum-likelihood pretraining objective, and autoregressive commitment under exposure bias -- as candidate failure surfaces, justify their separability rather than assuming it, and specify an attribution procedure requiring only sampling access: an ordered set of three interventions on prefix, context, and frequency competition, together with a validation design based on independent annotation and a classifier baseline. We state five falsifiable predictions and identify competing accounts each would discriminate against. We analyse how instruction tuning, RLHF, DPO, retrieval augmentation, scale, and calibration bear on the argument. We execute a direct, pre-registered test of the commitment prediction (P3) across three model families: substituting a correct continuation at the point of divergence reduces downstream failing claims by 46.7 percentage points relative to baseline (p<10^-9). However, a wrong-fact substitution reduces errors at a statistically indistinguishable rate, and the model answers correctly in isolation on only 2.2% of items where substitution succeeded -- a genuine partial result rather than a confirmation. Dataset pathologies amplify each component without originating failure independently, supporting an asymmetric-dependence claim: components are necessary intermediaries for data-induced failure, but data defects are not necessary for component-induced failure.
Abstract:Large language models produce fluent, confident, factually wrong output. Existing taxonomies classify these failures by output type -- intrinsic versus extrinsic, faithfulness versus factuality -- but say nothing about which computational component produced a given failure. We ask what would be required to attribute an individual hallucination to a specific component of the decoder-only stack. We treat three components -- self-attention's associative retrieval, the maximum-likelihood pretraining objective, and autoregressive commitment under exposure bias -- as candidate failure surfaces, justify their separability rather than assuming it, and specify an attribution procedure requiring only sampling access: an ordered set of three interventions on prefix, context, and frequency competition, together with a validation design based on independent annotation and a classifier baseline. We state five falsifiable predictions and identify competing accounts each would discriminate against. We analyse how instruction tuning, RLHF, DPO, retrieval augmentation, scale, and calibration bear on the argument. We execute a direct, pre-registered test of the commitment prediction (P3) across three model families: substituting a correct continuation at the point of divergence reduces downstream failing claims by 46.7 percentage points relative to baseline (p<10^-9). However, a wrong-fact substitution reduces errors at a statistically indistinguishable rate, and the model answers correctly in isolation on only 2.2% of items where substitution succeeded -- a genuine partial result rather than a confirmation. Dataset pathologies amplify each component without originating failure independently, supporting an asymmetric-dependence claim: components are necessary intermediaries for data-induced failure, but data defects are not necessary for component-induced failure.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2606.07537 [cs.CL]
(or arXiv:2606.07537v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.07537 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 347 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 18 sept. 2026
From Architecture to Output: Structural Origins of Hallucination in Large Language Models and the Amplifying Role of Data
arXiv · 12 iun. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026