aiqbench analysis
AfriSwitch exposes a critical gap in current speech recognition models by revealing that zero-shot performance on African languages drops significantly when handling natural code-switching, far worse than monolingual benchmarks suggest. The study demonstrates that targeted training for specific linguistic mixing patterns is more effective than simply increasing model scale or nominal language coverage. This benchmark provides essential data for improving ASR systems to handle the complex, in-the-wild bilingual realities of African conversation.
AfriSwitch: A Benchmark for In-the-Wild African Code-Switched Speech Recognition
arXiv:2608.26434v1 Announce Type: new Abstract: Code-switching is pervasive in bilingual African conversation, yet most ASR systems assume monolingual input and are evaluated on curated monolingual benchmarks. We present AfriSwitch, a 61.36-hour human-transcribed benchmark of in-the-wild code-switched speech spanning 16 African languages and language varieties, released with switch-level English span tags, perutterance Code-Mixing Index (CMI), and switch-point counts. Corpus statistics show that mixing behaviour varies widely across African languages along two largely independent axes: how often speakers alternate, and how balanced the mixture is. No single scalar captures how code-switched a language is. Benchmarking five open and commercial multilingual ASR systems zero-shot yields word error rates far above published monolingual figures for the same languages, with the best system averaging 35.93% WER and no system falling below 24% on any language. Africa-targeted training, not model scale or nominal language coverage, best predicts performance.
Abstract:Code-switching is pervasive in bilingual African conversation, yet most ASR systems assume monolingual input and are evaluated on curated monolingual benchmarks. We present AfriSwitch, a 61.36-hour human-transcribed benchmark of in-the-wild code-switched speech spanning 16 African languages and language varieties, released with switch-level English span tags, perutterance Code-Mixing Index (CMI), and switch-point counts. Corpus statistics show that mixing behaviour varies widely across African languages along two largely independent axes: how often speakers alternate, and how balanced the mixture is. No single scalar captures how code-switched a language is. Benchmarking five open and commercial multilingual ASR systems zero-shot yields word error rates far above published monolingual figures for the same languages, with the best system averaging 35.93% WER and no system falling below 24% on any language. Africa-targeted training, not model scale or nominal language coverage, best predicts performance.
Computation and Language (cs.CL)
arXiv:2608.26434 [cs.CL]
(or arXiv:2608.26434v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.26434 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 232 words
Related coverage
Beyond Word Error Rate: A Switch Aware Evaluation of ASR and Audio Language Models on English Yoruba Code-Switched Speech
arXiv · Sep 12, 2026
The Open ASR Leaderboard Adds Its First Global South Language
Hugging Face · Aug 28, 2026
Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech
Hugging Face · Jun 9, 2026
[AINews] Claude Opus 5.5, the new default model for AINews — and everybody cuts prices 40-50%
Latent Space · Sep 23, 2026