analiza aiqbench
This research identifies a human-like "attention distraction" mechanism in multimodal models that leads to object hallucinations and proposes a training-free correction method called AFIP. By enriching cross-head attention and enhancing dynamic historical attention, the approach improves visual grounding without increasing model complexity. The work offers a significant theoretical and practical advance for reducing hallucinations in vision-language systems through better attention management.
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
arXiv:2605.24602v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training.
Abstract:Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2605.24602 [cs.CV]
(or arXiv:2605.24602v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2605.24602 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 232 cuvinte
Subiecte conexe
SCoNE: Selective Context-aware Neuron Editing for Robust Retrieval-Augmented Generation
arXiv · 2 sept. 2026
When the Wrong Key Wins: Understanding and Detecting Hallucinations in LLMs
arXiv · 15 sept. 2026
The State of Context Engineering
Drew Breunig · 24 oct. 2026
The Geometry of Representational Failures in Vision Language Models
arXiv · 8 iun. 2026