analiza aiqbench
This research introduces Layer-Weighted Vector Steering (LWVS), a training-free method that significantly reduces hallucinations in Large Audio-Language Models by leveraging silence-anchored contrastive techniques. By identifying influential internal layers, LWVS boosts recall on audio QA tasks while preserving general audio understanding across major models like Gemma and Qwen. The approach represents a critical step toward making audio AI more reliable and grounded in actual sound data.
Silence is Golden: Mitigating Hallucinations in Large Audio-Language Models via Layer-Weighted Vector Steering
arXiv:2510.12851v2 Announce Type: replace-cross Abstract: Large Audio-Language Models (LALMs) excel in Audio QA but often suffer from hallucinations ungrounded in the audio. To our knowledge, we are the first to propose applying vector steering to the audio domain to mitigate this. Unlike text-based steering, our silence-anchored contrastive approach steers the model away from hallucinations by contrasting active audio against a silent baseline. Probing internal states reveals a strong correlation between specific layer representations and output correctness. Leveraging this, we introduce Layer-Weighted Vector Steering (LWVS), a training-free intervention that increases steering strength at influential layers. On the Audio Hallucination QA dataset, LWVS significantly outperforms baselines, boosting Recall on the Gemma model by 15.6% (53.4% to 69.0%). Crucially, MMAU benchmark tests confirm LWVS preserves and even enhances general audio understanding, achieving an 8% relative accuracy increase on the Qwen model (54.8% to 59.2%).
Abstract:Large Audio-Language Models (LALMs) excel in Audio QA but often suffer from hallucinations ungrounded in the audio. To our knowledge, we are the first to propose applying vector steering to the audio domain to mitigate this. Unlike text-based steering, our silence-anchored contrastive approach steers the model away from hallucinations by contrasting active audio against a silent baseline. Probing internal states reveals a strong correlation between specific layer representations and output correctness. Leveraging this, we introduce Layer-Weighted Vector Steering (LWVS), a training-free intervention that increases steering strength at influential layers. On the Audio Hallucination QA dataset, LWVS significantly outperforms baselines, boosting Recall on the Gemma model by 15.6% (53.4% to 69.0%). Crucially, MMAU benchmark tests confirm LWVS preserves and even enhances general audio understanding, achieving an 8% relative accuracy increase on the Qwen model (54.8% to 59.2%).
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
arXiv:2510.12851 [cs.SD]
(or arXiv:2510.12851v2 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2510.12851 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 235 cuvinte
Subiecte conexe
MISHAP-Bench: A Hallucination Benchmark for Large Audio-Language Models
arXiv · 30 sept. 2026
Natera’s intelligent appointment scheduling with Amazon Bedrock AgentCore
AWS · 26 aug. 2026
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
arXiv · 30 sept. 2026
TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models
arXiv · 9 sept. 2026