aiqbench analysis
This research introduces scalable, reference-free metrics for evaluating forced alignment in speech processing using self-supervised representations, addressing the bottleneck of manual timestamp annotation. By validating these methods across 85 languages, the study demonstrates that acoustic consistency and phoneme cluster agreement effectively correlate with traditional quality measures without requiring ground-truth timestamps. This advancement facilitates large-scale, multilingual analysis essential for training robust AI speech models.
Phoneme- and Word-Level Metrics Using Self-Supervised Speech Representations for Forced Alignment Evaluation
arXiv:2608.28508v1 Announce Type: new Abstract: Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at https://github.com/mahesh-ak/forced-aligner-metrics.
Abstract:Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at this https URL.
Computation and Language (cs.CL)
arXiv:2608.28508 [cs.CL]
(or arXiv:2608.28508v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.28508 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 256 words
Related coverage
Iterative Self-Learning for Expressive Text-to-Speech Synthesis
arXiv · Aug 18, 2026
Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations
arXiv · Jun 8, 2026
Evaluate Clinical ASR Models Faster with Agent Skills and NVIDIA Nemotron Speech
NVIDIA · Jun 9, 2026
The Open ASR Leaderboard Adds Its First Global South Language
Hugging Face · Aug 28, 2026