aiqbench analysis
This study investigates whether explicitly trained prosodic representations offer measurable gains in automatic speech recognition beyond the benefits of trainable fusion mechanisms alone. Using a frozen HuBERT backbone, researchers found that while the auxiliary representation is necessary for optimal performance, it provides no significant word error rate reduction compared to a parameter-matched control without the input. The findings suggest that the primary improvements in spontaneous speech ASR stem from the fusion strategy rather than the specific prosodic features themselves.
Does a prosody-trained representation help beyond trainable fusion? A parameter-matched study with frozen HuBERT
arXiv:2609.36754v1 Announce Type: cross Abstract: Explicit prosodic cues may help automatic speech recognition (ASR) of spontaneous speech, but auxiliary representations typically require additional trainable components, making it unclear whether gains come from the auxiliary information or the fusion mechanism. We address this using a frozen HuBERT backbone and a 64-dimensional representation trained to predict log F0, voicing, Delta log F0, log energy, and spectral tilt. We compare a frozen-backbone recognizer (Baseline), trainable fusion with zero auxiliary input (Null), and the same fusion supplied with the learned representation (Learned). Across Buckeye, Switchboard, and AMI IHM, Null reduces WER by 0.71-1.45 points over Baseline, whereas Learned differs from Null by +0.07, -0.09, and +0.00 points, with no significant differences. However, removing or mismatching the representation at inference increases Learned WER. Thus, Learned depends on the representation yet shows no measurable incremental WER benefit over the parameter-matched control.
Title:Does a prosody-trained representation help beyond trainable fusion? A parameter-matched study with frozen HuBERT
Abstract:Explicit prosodic cues may help automatic speech recognition (ASR) of spontaneous speech, but auxiliary representations typically require additional trainable components, making it unclear whether gains come from the auxiliary information or the fusion mechanism. We address this using a frozen HuBERT backbone and a 64-dimensional representation trained to predict log F0, voicing, Delta log F0, log energy, and spectral tilt. We compare a frozen-backbone recognizer (Baseline), trainable fusion with zero auxiliary input (Null), and the same fusion supplied with the learned representation (Learned). Across Buckeye, Switchboard, and AMI IHM, Null reduces WER by 0.71-1.45 points over Baseline, whereas Learned differs from Null by +0.07, -0.09, and +0.00 points, with no significant differences. However, removing or mismatching the representation at inference increases Learned WER. Thus, Learned depends on the representation yet shows no measurable incremental WER benefit over the parameter-matched control.
Audio and Speech Processing (eess.AS); Computation and Language (cs.CL)
arXiv:2609.36754 [eess.AS]
(or arXiv:2609.36754v1 [eess.AS] for this version)
https://doi.org/10.48550/arXiv.2609.36754 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 254 words
