Adapting Foundation ASR Models to Dysarthric Speech: A Case Study

Abstract:Automatic speech recognition (ASR) systems often perform poorly in dysarthric speech, limiting their usefulness to affected speakers in everyday communication. This paper presents a personalized ASR system for a dysarthric speaker, built by adapting a foundation ASR model to speaker-specific data. Using the TEQST tool, we collected 92 hours of read speech and later added 8.8 hours of user corrections gathered through a deployed mobile application. Starting from Whisper, fine-tuning reduced word error rate to 15.8% with only 1.4 hours of adaptation data on the read test set, reached 10.7% / 16.1% with 22.5 hours on the read and corrections test sets, respectively, and achieved the best result of 9.7% when using all available data including the corrections on the read test set and 7.8% on the corrections test set. Using LoRA adaptation and/or Qwen3-ASR as foundation model performed worse in this setting. The results show that personalized fine-tuning can make foundation ASR models substantially more effective for dysarthric speech and suitable for practical deployment.

Computation and Language (cs.CL)

arXiv:2606.31722 [cs.CL]

(or arXiv:2606.31722v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.31722 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.