analiza aiqbench
This research investigates the cross-language generalization capabilities of deep neural speaker embeddings, demonstrating that they align closely with human perception in bilingual Cantonese-English contexts. By leveraging shared acoustic cues and language-sensitive weighting, the study provides empirical evidence for human-machine alignment in voice similarity processing across different linguistic boundaries. The findings advance our understanding of the acoustic foundations required for robust, multilingual speech recognition systems.
Human-like cross-language generalisation in deep neural speaker embeddings and its acoustic foundations
arXiv:2611.05384v1 Announce Type: new Abstract: Generalisability is a central question for deep neural networks, including the cross-language generalisation of speaker embeddings. This study examined whether deep neural speaker embeddings generalise across languages and how such generalisation relates to human perception and acoustic structure. Using Cantonese--English bilingual speech presented in within- and cross-language conditions, we derived similarity from 18 state-of-the-art models and compared it with human ratings and acoustic features using linear mixed models and representational similarity analysis. Embeddings exhibited cross-language generalisation that broadly paralleled human behaviour, reflecting accurate identity discrimination and sensitivity to language switching. This generalisation relied on largely shared acoustic cues with language-sensitive weighting, mirroring human patterns. Together, these findings provide converging evidence for human--machine alignment in cross-language voice similarity processing.
Abstract:Generalisability is a central question for deep neural networks, including the cross-language generalisation of speaker embeddings. This study examined whether deep neural speaker embeddings generalise across languages and how such generalisation relates to human perception and acoustic structure. Using Cantonese--English bilingual speech presented in within- and cross-language conditions, we derived similarity from 18 state-of-the-art models and compared it with human ratings and acoustic features using linear mixed models and representational similarity analysis. Embeddings exhibited cross-language generalisation that broadly paralleled human behaviour, reflecting accurate identity discrimination and sensitivity to language switching. This generalisation relied on largely shared acoustic cues with language-sensitive weighting, mirroring human patterns. Together, these findings provide converging evidence for human--machine alignment in cross-language voice similarity processing.
This manuscript has been accepted for presentation at Interspeech 2026. Code is available at this https URL. The archived release is available on Zenodo at this https URL
Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
arXiv:2611.05384 [cs.CL]
(or arXiv:2611.05384v1 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 165 cuvinte