analiza aiqbench
This research reveals a critical flaw in standard multi-task learning for dual-output second-language speech recognition, where shared encoder representations degrade surface transcription accuracy, particularly in English. The study identifies encoder-level entanglement as the root cause, noting that while Korean preserves distinct task representations, English models produce nearly identical ones that constrain the surface decoder. These findings provide a necessary direction for designing new MTL frameworks that mitigate entanglement to prevent performance loss in pronunciation tasks.
Multi-task Learning is Not Enough: Representational Entanglement in Dual-output Second Language Speech Recognition
arXiv:2606.06065v3 Announce Type: replace Abstract: Second-language (L2) speech recognition often requires transcriptions of pronunciations and intended meanings. Multi-task learning (MTL) is a natural approach because it assumes that shared representations benefit both outputs. However, this paper shows that this assumption does not hold across Korean and English. MTL improves meaning but degrades surface transcription, especially in English, where the degradation scales with surface-meaning divergence measured by Levenshtein edit distance. Encoder analysis links these patterns to encoder-level entanglement, with Korean preserving distinct task representations while English produces nearly identical ones. Cross-task decoder analysis shows that the meaning dual-output decoder adapts with a unique representation, while the surface dual-output decoder remains constrained by the encoder. These findings motivate the design of MTL frameworks that mitigate encoder-level entanglement to reduce surface degradation in dual-output L2 automatic speech recognition.
Abstract:Second-language (L2) speech recognition often requires transcriptions of pronunciations and intended meanings. Multi-task learning (MTL) is a natural approach because it assumes that shared representations benefit both outputs. However, this paper shows that this assumption does not hold across Korean and English. MTL improves meaning but degrades surface transcription, especially in English, where the degradation scales with surface-meaning divergence measured by Levenshtein edit distance. Encoder analysis links these patterns to encoder-level entanglement, with Korean preserving distinct task representations while English produces nearly identical ones. Cross-task decoder analysis shows that the meaning dual-output decoder adapts with a unique representation, while the surface dual-output decoder remains constrained by the encoder. These findings motivate the design of MTL frameworks that mitigate encoder-level entanglement to reduce surface degradation in dual-output L2 automatic speech recognition.
Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)
arXiv:2606.06065 [cs.CL]
(or arXiv:2606.06065v3 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 149 cuvinte
Subiecte conexe
Multi-task Learning is Not Enough: Representational Entanglement in Dual-output Second Language Speech Recognition
arXiv · 8 iun. 2026
Multi-task Learning is Not Enough: Representational Entanglement in Dual-output Second Language Speech Recognition
arXiv · 5 iun. 2026
Reflections on unlearning and inoculation
LessWrong · 20 sept. 2026
Self-Modeling Interventions Modulate Emergent Misalignment
LessWrong · 5 oct. 2026