analiza aiqbench
This research introduces a projector-based LLM-ASR framework that leverages a Mixture of Experts architecture to enhance cross-lingual adaptability while using a Continuous Integrate-and-Fire mechanism for dynamic downsampling. The approach demonstrates substantial performance gains over strong baselines, offering a viable path toward more robust and generalizable multilingual speech recognition systems. By addressing modality alignment challenges directly, this work provides a technical foundation for scaling LLMs in diverse audio processing tasks.
Enhancing Multilingual LLM-based ASR with Mixture of Experts and Dynamic Downsampling
arXiv:2606.10439v1 Announce Type: cross Abstract: The rapid progress of large language models (LLMs) has opened up a new frontier for automatic speech recognition (ASR), making their effective integration a critical and challenging research direction. To this end, this work proposes a projector-based LLM-ASR framework targeting the key challenges of multilingual generalization and modality alignment. Our approach incorporates a Mixture of Experts (MoE) architecture to improve cross-lingual adaptability, and a Continuous Integrate-and-Fire (CIF) mechanism for dynamic downsampling and modality alignment. Experimental results show that the combination of these components yields substantial performance improvements, surpassing strong baseline models. The proposed method represents a step toward building more accurate, robust, and generalizable LLM-based ASR systems.
Abstract:The rapid progress of large language models (LLMs) has opened up a new frontier for automatic speech recognition (ASR), making their effective integration a critical and challenging research direction. To this end, this work proposes a projector-based LLM-ASR framework targeting the key challenges of multilingual generalization and modality alignment. Our approach incorporates a Mixture of Experts (MoE) architecture to improve cross-lingual adaptability, and a Continuous Integrate-and-Fire (CIF) mechanism for dynamic downsampling and modality alignment. Experimental results show that the combination of these components yields substantial performance improvements, surpassing strong baseline models. The proposed method represents a step toward building more accurate, robust, and generalizable LLM-based ASR systems.
Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
arXiv:2606.10439 [cs.SD]
(or arXiv:2606.10439v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2606.10439 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1109/ICASSP55912.2026.11464266 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 219 cuvinte
Subiecte conexe
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · 21 mai 2026
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · 24 iul. 2026
Risk-Aware Decision-Making for Autonomous Overtaking: A World Model-Based Mixture-of-Experts Framework
arXiv · 2 sept. 2026
STHMoE: Hypergraph-Enhanced Heterogeneous Dependency Coordination for LLM-Based Urban Traffic Data Forecasting
arXiv · 16 sept. 2026