aiqbench analysis
This research advances speech enhancement by leveraging continuous latent representations from neural audio codecs within a masked autoregressive framework, offering a flexible trade-off between audio quality and computational efficiency. By utilizing the DAC codec's continuous domain alongside a Conformer model, the study demonstrates that moving beyond discrete tokenization can significantly improve speech intelligibility without altering the core training architecture.
Masked Autoregressive Speech Enhancement with Continuous Neural Audio Codec Representations
arXiv:2609.03940v1 Announce Type: cross Abstract: Most previous work on speech enhancement (SE) based on masked generative modeling relied on discrete token representations of audio signals, obtained using neural audio codecs (NACs). However, a recent study has shown that continuous latent representations of NACs can be advantageous for SE in terms of speech quality and intelligibility. In this work, we propose masked autoregressive SE (MARSE), a method for SE based on iterative decoding of masked clean speech frames using continuous NAC representations of speech. In particular, we investigate a set of different decoding policies, ceteris paribus, that is, using the same DNN (a Conformer model), the same NAC (the DAC codec) and the same training setup. The results show that MARSE enables a flexible trade-off between SE performance and computational cost. Audio examples and code are available online.
Abstract:Most previous work on speech enhancement (SE) based on masked generative modeling relied on discrete token representations of audio signals, obtained using neural audio codecs (NACs). However, a recent study has shown that continuous latent representations of NACs can be advantageous for SE in terms of speech quality and intelligibility. In this work, we propose masked autoregressive SE (MARSE), a method for SE based on iterative decoding of masked clean speech frames using continuous NAC representations of speech. In particular, we investigate a set of different decoding policies, ceteris paribus, that is, using the same DNN (a Conformer model), the same NAC (the DAC codec) and the same training setup. The results show that MARSE enables a flexible trade-off between SE performance and computational cost. Audio examples and code are available online.
Sound (cs.SD); Artificial Intelligence (cs.AI)
arXiv:2609.03940 [cs.SD]
(or arXiv:2609.03940v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.03940 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 228 words
Related coverage
What’s New for Game Developers: DLSS 5 with 3D-Guided Neural Rendering, NVIDIA ACE Updates, and New RTX Kit Capabilities
NVIDIA · Sep 22, 2026
DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis
arXiv · Sep 16, 2026
Efficient Audio-Visual Speech Enhancement via Reinforcement Learning-Based Neural Architecture Search
arXiv · Oct 12, 2026
Test-time adaptation for speech enhancement with an autoregressive speech prior
arXiv · Sep 5, 2026