Quantifying Cross-Lingual Transfer in Paralinguistic Speech Tasks

Abstract:Paralinguistic speech tasks are often considered relatively language-agnostic, as they rely on extralinguistic acoustic cues rather than lexical content. However, prior studies report performance degradation under cross-lingual conditions, indicating non-negligible language dependence. Still, these studies typically focus on isolated language pairs or task-specific settings, limiting comparability and preventing a systematic assessment of task-level language dependence. We introduce the Cross-Lingual Transfer Matrix (CLTM), a systematic method to quantify cross-lingual interactions between pairs of languages within a given task. We apply the CLTM to two paralinguistic tasks, gender identification and speaker verification, using a multilingual HuBERT-based encoder, to analyze how donor-language data affects target-language performance during fine-tuning. Our results reveal distinct transfer patterns across tasks and languages, reflecting systematic, language-dependent effects.

Audio and Speech Processing (eess.AS); Computation and Language (cs.CL)

arXiv:2603.08231 [eess.AS]

(or arXiv:2603.08231v2 [eess.AS] for this version)

https://doi.org/10.48550/arXiv.2603.08231 Focus to learn more arXiv-issued DOI via DataCite

Proceedings of Interspeech 2026, pp. 2297--2302

https://doi.org/10.21437/Interspeech.2026-2745 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.