analiza aiqbench
This research addresses hallucinations in large audio-language models by advancing contrastive decoding through adaptive, structured perturbations rather than blunt masking techniques. By dynamically selecting optimal transformations based on model hidden states and task requirements, the method significantly improves accuracy without requiring additional training data or heavy computational overhead. The findings offer a practical pathway for enhancing the reliability of multimodal AI systems that process acoustic evidence alongside linguistic priors.
Adaptive Perturbation Selection for Contrastive Audio Decoding
arXiv:2607.00247v2 Announce Type: replace-cross Abstract: Large audio-language models (LALMs) frequently hallucinate by overriding acoustic evidence with language priors. While contrastive decoding (CD) offers training-free mitigation, existing methods rely on blunt perturbations like masking or noise, leaving structured audio transformations unexplored. We explore this design space by evaluating a diverse library of targeted audio perturbations and adaptively selecting the optimal negative branch for each task and example. First, we improve upon earlier prompt engineering by showing that a simple binary yes/no constraint reduces the model's tendency to falsely confirm absent audio features. Second, evaluating our library across temporal, spectral, frequency, and amplitude domains reveals that optimal transformations are highly task-dependent; for instance, reversing the audio array disrupts temporal coherence, raising accuracy on the temporal order task from 74.7% to 81.4%. Finally, we trained a light-weight perturbation selector on model hidden states to dynamically route negative branches, yielding an additional +4.3% gain on the existence task.
Abstract:Large audio-language models (LALMs) frequently hallucinate by overriding acoustic evidence with language priors. While contrastive decoding (CD) offers training-free mitigation, existing methods rely on blunt perturbations like masking or noise, leaving structured audio transformations unexplored. We explore this design space by evaluating a diverse library of targeted audio perturbations and adaptively selecting the optimal negative branch for each task and example. First, we improve upon earlier prompt engineering by showing that a simple binary yes/no constraint reduces the model's tendency to falsely confirm absent audio features. Second, evaluating our library across temporal, spectral, frequency, and amplitude domains reveals that optimal transformations are highly task-dependent; for instance, reversing the audio array disrupts temporal coherence, raising accuracy on the temporal order task from 74.7% to 81.4%. Finally, we trained a light-weight perturbation selector on model hidden states to dynamically route negative branches, yielding an additional +4.3% gain on the existence task.
Sound (cs.SD); Artificial Intelligence (cs.AI)
arXiv:2607.00247 [cs.SD]
(or arXiv:2607.00247v2 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2607.00247 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 243 cuvinte
Subiecte conexe
X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
arXiv · 12 sept. 2026
TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models
arXiv · 9 sept. 2026
ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding
arXiv · 7 sept. 2026
[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026