aiqbench analysis
This research demonstrates that fine-tuned self-supervised models like Wav2Vec2 and HuBERT can generalize to recognize rare phonemes, such as click consonants in Khoisan languages, with higher accuracy than non-click sounds. The findings suggest that self-supervised learning effectively mitigates data scarcity issues for low-resource languages, enabling robust recognition of typologically uncommon speech sounds without extensive labeled datasets.
Pretrained self-supervised speech models can recognize unseen consonants
arXiv:2606.11542v1 Announce Type: new Abstract: Modern pretrained self-supervised automatic speech recognition models are trained on large-scale audio data to encode speech into contextualized representations. However, their training data are heavily skewed toward high-resource languages with little data from low-resource languages, raising concerns about the potential underrepresentation of typologically uncommon speech sounds such as click consonants primarily found in Khoisan languages. This leads to our central research question: Can these models recognize click consonants as accurately as other speech sounds? To address this question, we fine-tune and compare pretrained self-supervised speech models (Wav2Vec2 and HuBERT) on data from two click-rich Khoisan languages (G|ui and West !Xoon). Our results reveal that the fine-tuned models consistently recognize clicks more accurately than non-clicks, suggesting that self-supervision enables generalization across human speech sounds including rare phonemes.
Abstract:Modern pretrained self-supervised automatic speech recognition models are trained on large-scale audio data to encode speech into contextualized representations. However, their training data are heavily skewed toward high-resource languages with little data from low-resource languages, raising concerns about the potential underrepresentation of typologically uncommon speech sounds such as click consonants primarily found in Khoisan languages. This leads to our central research question: Can these models recognize click consonants as accurately as other speech sounds? To address this question, we fine-tune and compare pretrained self-supervised speech models (Wav2Vec2 and HuBERT) on data from two click-rich Khoisan languages (G|ui and West !Xoon). Our results reveal that the fine-tuned models consistently recognize clicks more accurately than non-clicks, suggesting that self-supervision enables generalization across human speech sounds including rare phonemes.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.11542 [cs.CL]
(or arXiv:2606.11542v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.11542 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 224 words
Related coverage
Pretrained self-supervised speech models can recognize unseen consonants
arXiv · Oct 10, 2026
From Self-Supervised Speech Models to Mixture-of-Experts for Robust Anti-Spoofing
arXiv · Jun 15, 2026
CRAF: Cross-View Residual-Aware Fusion for Deepfake Speech Detection
arXiv · Sep 16, 2026
ParaBridge: Bridging Paralinguistic Perception and Dialogue Behavior in Speech Language Models
arXiv · Jun 10, 2026