aiqbench analysis
This research presents a lightweight method for adapting LLM-based automatic speech recognition systems to new domains using only text data, avoiding the performance degradation caused by standard fine-tuning. By framing adaptation as a text denoising task, the approach successfully preserves critical cross-modal alignment between audio and text without requiring architectural changes or extra parameters. The technique achieves significant relative improvements over existing methods, offering an efficient solution for domain-specific ASR deployment.
Text-only adaptation in LLM-based ASR through text denoising
arXiv:2601.20900v4 Announce Type: replace-cross Abstract: Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet underexplored challenge. Standard fine-tuning of the LLM on the target domain text often disrupts the critical alignment between the speech and text modality learned by the projector, degrading performance. We introduce a novel text-only adaptation method that frames this process as a text denoising task. Our approach trains the LLM to recover clean transcripts from noisy inputs. This process effectively adapts the model to a target domain while preserving cross-modal alignment. Our solution is lightweight, requiring no architectural changes or additional parameters. Extensive evaluation on two datasets demonstrates up to 22.1% relative improvement, outperforming recent state-of-the-art text-only adaptation methods.
Abstract:Adapting large language model (LLM)-based automatic speech recognition (ASR) systems to new domains using text-only data is a significant yet underexplored challenge. Standard fine-tuning of the LLM on the target domain text often disrupts the critical alignment between the speech and text modality learned by the projector, degrading performance. We introduce a novel text-only adaptation method that frames this process as a text denoising task. Our approach trains the LLM to recover clean transcripts from noisy inputs. This process effectively adapts the model to a target domain while preserving cross-modal alignment. Our solution is lightweight, requiring no architectural changes or additional parameters. Extensive evaluation on two datasets demonstrates up to 22.1% relative improvement, outperforming recent state-of-the-art text-only adaptation methods.
Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
arXiv:2601.20900 [cs.SD]
(or arXiv:2601.20900v4 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2601.20900 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 222 words
Related coverage
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · Sep 4, 2026
VSA: Accelerating Video Diffusion Inference with Sparse Attention on AMD GPUs
AMD · Aug 4, 2026
Prototype-Rectified Iterative Self-supervised Manifold Denoising under Severe Acoustic Shift
arXiv · Aug 19, 2026
STAR: SpatioTemporal Adaptive Reward Allocation for Text-to-Image RL Post-Training
arXiv · Jun 17, 2026