aiqbench analysis
This research proposes integrating external language models directly into ASR parameters via LoRA arithmetic, eliminating the computational overhead typically associated with separate LM inference. The approach successfully enhances domain adaptation performance for large LLM-based speech systems without increasing memory footprint or latency. By merging knowledge rather than running parallel models, this method offers a more efficient pathway for scaling ASR capabilities with massive language models.
Merging the Knowledge of LLMs for Automatic Speech Recognition
arXiv:2609.15743v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems, trained on paired speech-text data, have been improved by leveraging language models (LMs) trained on text-only data. LM fusion methods such as shallow fusion and density ratio are well-established methods that incorporate external LMs during ASR decoding. However, they incur additional computational costs due to LM inference, which is particularly problematic for recent larger LMs. In this study, we propose incorporating external LMs via model merging. This method integrates the LMs directly into the parameters of an LLM-based ASR model, requiring no additional computational cost at inference. We formulate domain extension and transfer via arithmetic operations on LoRA parameters. Experimental evaluations were conducted for the domain adaptation of LLM-based ASR trained on CSJ and LibriSpeech. We show that our LM merging consistently improved the ASR performance in the target domains, without degrading inference speed or memory footprint.
Abstract:Automatic speech recognition (ASR) systems, trained on paired speech-text data, have been improved by leveraging language models (LMs) trained on text-only data. LM fusion methods such as shallow fusion and density ratio are well-established methods that incorporate external LMs during ASR decoding. However, they incur additional computational costs due to LM inference, which is particularly problematic for recent larger LMs. In this study, we propose incorporating external LMs via model merging. This method integrates the LMs directly into the parameters of an LLM-based ASR model, requiring no additional computational cost at inference. We formulate domain extension and transfer via arithmetic operations on LoRA parameters. Experimental evaluations were conducted for the domain adaptation of LLM-based ASR trained on CSJ and LibriSpeech. We show that our LM merging consistently improved the ASR performance in the target domains, without degrading inference speed or memory footprint.
Computation and Language (cs.CL)
arXiv:2609.15743 [cs.CL]
(or arXiv:2609.15743v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.15743 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 237 words
Related coverage
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
arXiv · Sep 17, 2026
Listen to the Latents: Self-Correcting Speech Recognition in Large Audio Language Models Through Hidden-State Interactions
arXiv · Sep 5, 2026
Machine Unlearning for Speech Question Answering in Large Audio-Language Models
arXiv · Sep 15, 2026
agentic-ger: terminology recovery in long-form speech using global context
arXiv · Sep 25, 2026