analiza aiqbench
This research demonstrates that explicitly training multilingual speech models to discriminate between languages during pretraining can significantly narrow the performance gap against monolingual baselines without sacrificing cross-language information sharing. By introducing language classifiers early in the HuBERT training process, the study achieves improvements in phonetic and lexical metrics while mitigating the data efficiency penalty typically associated with multilingual learning. These findings offer a practical architectural adjustment for developers aiming to optimize self-supervised speech models under constrained data budgets.
Language Discrimination Improves Linguistic Learning in Multilingual Speech Models
arXiv:2609.33345v1 Announce Type: new Abstract: Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total pretraining data budget they still fall short of monolingual models. We show that strengthening the model's ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing. Using a controlled English/French HuBERT setting, we test two interventions which strengthen language discrimination: an auxiliary language classifier and per-language k-means targets. Across interventions, continuous-feature phone discrimination error (phone-ABX, lower is better) decreases from 11.6% in the bilingual baseline to 10.4% (monolingual: 10.8%), while lexical performance (sWUGGY, higher is better) increases from 52.1% to 56.7% (monolingual: 58.5%) and prosodic performance (ProsAudit, lexical subtask, higher is better) from 68.9% to 72.9% (monolingual: 72.6%). Across HuBERT training stages, the strongest gains on most linguistic measures occur when language discrimination is introduced in the first iteration, whereas later or repeated interventions yield smaller improvements and are accompanied by increased language-wise segregation. These results support a causal role for language discrimination in reducing the additional cost of multilingual learning.
Abstract:Multilingual self-supervised speech models can benefit from sharing information across languages, but under a matched total pretraining data budget they still fall short of monolingual models. We show that strengthening the model's ability to discriminate languages during pretraining reduces and, on some measures, closes this multilingual gap on continuous phonetic and higher-level linguistic measures, while preserving substantial cross-language sharing. Using a controlled English/French HuBERT setting, we test two interventions which strengthen language discrimination: an auxiliary language classifier and per-language k-means targets. Across interventions, continuous-feature phone discrimination error (phone-ABX, lower is better) decreases from 11.6% in the bilingual baseline to 10.4% (monolingual: 10.8%), while lexical performance (sWUGGY, higher is better) increases from 52.1% to 56.7% (monolingual: 58.5%) and prosodic performance (ProsAudit, lexical subtask, higher is better) from 68.9% to 72.9% (monolingual: 72.6%). Across HuBERT training stages, the strongest gains on most linguistic measures occur when language discrimination is introduced in the first iteration, whereas later or repeated interventions yield smaller improvements and are accompanied by increased language-wise segregation. These results support a causal role for language discrimination in reducing the additional cost of multilingual learning.
Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
arXiv:2609.33345 [cs.CL]
(or arXiv:2609.33345v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.33345 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 286 cuvinte
Subiecte conexe
Language Discrimination Improves Linguistic Learning in Multilingual Speech Models
Apple · 2 oct. 2026
Tiny Aya in the wild: What people are building with open multilingual AI
Cohere · 14 iul. 2026
MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition
arXiv · 28 iul. 2026
MauBERT: Universal Phonetic Inductive Biases for Few-Shot Acoustic Units Discovery
arXiv · 26 aug. 2026