aiqbench analysis
This paper introduces a novel training criterion for energy-based models in text-to-speech, specifically addressing the limitations of existing methods like noise contrastive estimation and sliced score matching regarding first-order optimization schemes. By aligning learned scores with the inference requirements of diffusion models, the work offers a more robust approach to training EBMs that could improve audio generation fidelity.
Score-Based Training for Energy-Based TTS Models
arXiv:2505.13771v2 Announce Type: replace-cross Abstract: Noise contrastive estimation (NCE) is a popular method for training energy-based models (EBM) with intractable normalisation terms. The key idea of NCE is to learn by comparing unnormalised log-likelihoods of the reference and noisy samples, thus avoiding explicitly computing normalisation terms. However, NCE critically relies on the quality of noisy samples. Recently, sliced score matching (SSM) has been popularised by closely related diffusion models (DM). Unlike NCE, SSM learns a gradient of log-likelihood, or score, by learning distribution of its projections on randomly chosen directions. However, both NCE and SSM disregard the form of log-likelihood function, which is problematic given that EBMs and DMs make use of first-order optimisation during inference. This paper proposes a new criterion that learns scores more suitable for first-order schemes. Experiments contrasts these approaches for training EBMs.
Abstract:Noise contrastive estimation (NCE) is a popular method for training energy-based models (EBM) with intractable normalisation terms. The key idea of NCE is to learn by comparing unnormalised log-likelihoods of the reference and noisy samples, thus avoiding explicitly computing normalisation terms. However, NCE critically relies on the quality of noisy samples. Recently, sliced score matching (SSM) has been popularised by closely related diffusion models (DM). Unlike NCE, SSM learns a gradient of log-likelihood, or score, by learning distribution of its projections on randomly chosen directions. However, both NCE and SSM disregard the form of log-likelihood function, which is problematic given that EBMs and DMs make use of first-order optimisation during inference. This paper proposes a new criterion that learns scores more suitable for first-order schemes. Experiments contrasts these approaches for training EBMs.
5 pages, 3 figures, 4 tables. Accepted to Interspeech 2025
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
arXiv:2505.13771 [cs.SD]
(or arXiv:2505.13771v2 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2505.13771 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 241 words
Related coverage
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · Sep 14, 2026
Decision-Aware Training for Sample-Based Generative Models
arXiv · Oct 5, 2026
Composing Flow-Matching Energies with Known Physics: Generation, OOD Detection, and Inversion on PDE Fields
arXiv · Aug 20, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026