analiza aiqbench
The paper examines why irrelevant earlier audio in speech LLMs can still steer answers even when the correct source is retained. Using causal interventions on MOSS-Audio and MiMo-Audio, the authors find that conversational memory fails not only from poor retrieval but also from weak control over how retained information enters the answer computation. For developers of multi-turn voice assistants, this suggests that better memory storage alone won't fix context contamination, and that multiple-choice evaluations may understate these errors.
The Right Source Is Not Enough: Understanding History Interference in Speech-LLM Memory
arXiv:2611.05124v1 Announce Type: cross Abstract: Speech large language models retain audio history to support multi-turn conversations. Yet remembering an event does not ensure that its information is used only when relevant. We investigate why irrelevant historical audio changes answers even when the queried event remains represented. Controlled experiments on MOSS-Audio and MiMo-Audio show that historical values direct errors toward specific competing answers, beyond a generic effect of context length. Causal interventions distinguish two paths. In open-ended question answering, most history-value errors disappear when subsequent computation cannot directly access the irrelevant source. In multiple-choice answering, many errors persist: early historical influence is carried by question-candidate states even after direct source access is blocked. Low-dimensional interventions further separate source-role information from answer-value information. Their effects peak at different depths, and changing the source-role signal does not eliminate the causal influence of competing values. On naturally occurring errors, strengthening the queried role and counteracting the historical value each improve answers; their joint intervention gives the greatest recovery. These findings support a failure of downstream evidence use beyond source identification alone: source-role information can influence a decision without exclusively controlling which historical values affect it. Reliable conversational memory therefore requires not only retention and retrieval, but also control over how retained information enters answer computation.
Abstract:Speech large language models retain audio history to support multi-turn conversations. Yet remembering an event does not ensure that its information is used only when relevant. We investigate why irrelevant historical audio changes answers even when the queried event remains represented. Controlled experiments on MOSS-Audio and MiMo-Audio show that historical values direct errors toward specific competing answers, beyond a generic effect of context length. Causal interventions distinguish two paths. In open-ended question answering, most history-value errors disappear when subsequent computation cannot directly access the irrelevant source. In multiple-choice answering, many errors persist: early historical influence is carried by question-candidate states even after direct source access is blocked. Low-dimensional interventions further separate source-role information from answer-value information. Their effects peak at different depths, and changing the source-role signal does not eliminate the causal influence of competing values. On naturally occurring errors, strengthening the queried role and counteracting the historical value each improve answers; their joint intervention gives the greatest recovery. These findings support a failure of downstream evidence use beyond source identification alone: source-role information can influence a decision without exclusively controlling which historical values affect it. Reliable conversational memory therefore requires not only retention and retrieval, but also control over how retained information enters answer computation.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2611.05124 [cs.CL]
(or arXiv:2611.05124v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2611.05124 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 304 cuvinte
Subiecte conexe
Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models
arXiv · 1 oct. 2026
Uncovering Cross-Objective Interference in Multi-Objective Alignment
arXiv · 7 oct. 2026
Continual Adaptation Across Modalities: Learning and Forgetting in Unified Multimodal Models
arXiv · 30 oct. 2026
AuEmoChat: Authentic Emotion Understanding and Rendering for Conversational Speech Synthesis
arXiv · 3 aug. 2026