aiqbench analysis
This research addresses the quality disparity in text-to-speech systems for low-resource languages by applying LoRA fine-tuning to the VoxCPM2 model. The study demonstrates that a single shared adapter can significantly improve native-speaker ratings for Khmer while revealing that adaptation offers limited benefit for well-covered languages like Korean. Notably, the findings highlight a divergence between standard training loss metrics and human perceptual scores regarding optimal adapter rank.
Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean
arXiv:2606.26618v2 Announce Type: replace Abstract: Large pretrained text-to-speech (TTS) models sound almost human for well-resourced languages, but much worse for languages that are rare in their training data. We study this quality gap for Khmer and Korean using VoxCPM2, a 2.4B parameter, tokenizer-free TTS model that joins a MiniCPM-4 language-model backbone with a flow-matching diffusion decoder. We build one shared, language-tagged corpus of 25.5 hours after cleaning and adapt VoxCPM2 with a single Low-Rank Adaptation (LoRA) adapter, trained on both languages at once and added to both the language model and the decoder. The adapter is zero-initialized, so training starts exactly at the original zero-shot model. In native-speaker listening tests, the Khmer Mean Opinion Score (MOS) rises from 3.85 to 4.23 with the best adapter, rank 64. This gain is highly significant under a paired Wilcoxon test with p < 0.001, and it is achieved while training only 0.19 to 3.03 percent of the parameters. Two findings stand out. First, the training loss and human ratings disagree on the best rank. The loss is lowest at rank 128, but MOS peaks at rank 64. Second, the same adapter gives no significant gain for Korean, which the base model already covers well, and a high rank even hurts quality. This shows that adaptation helps mainly where the base model is truly weak.
Abstract:Large pretrained text-to-speech (TTS) models sound almost human for well-resourced languages, but much worse for languages that are rare in their training data. We study this quality gap for Khmer and Korean using VoxCPM2, a 2.4B parameter, tokenizer-free TTS model that joins a MiniCPM-4 language-model backbone with a flow-matching diffusion decoder. We build one shared, language-tagged corpus of 25.5 hours after cleaning and adapt VoxCPM2 with a single Low-Rank Adaptation (LoRA) adapter, trained on both languages at once and added to both the language model and the decoder. The adapter is zero-initialized, so training starts exactly at the original zero-shot model. In native-speaker listening tests, the Khmer Mean Opinion Score (MOS) rises from 3.85 to 4.23 with the best adapter, rank 64. This gain is highly significant under a paired Wilcoxon test with p < 0.001, and it is achieved while training only 0.19 to 3.03 percent of the parameters. Two findings stand out. First, the training loss and human ratings disagree on the best rank. The loss is lowest at rank 128, but MOS peaks at rank 64. Second, the same adapter gives no significant gain for Korean, which the base model already covers well, and a high rank even hurts quality. This shows that adaptation helps mainly where the base model is truly weak.
Computation and Language (cs.CL)
arXiv:2606.26618 [cs.CL]
(or arXiv:2606.26618v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.26618 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 309 words
