Improved off-policy training of diffusion samplers

Abstract:We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at this https URL as a base for future work on diffusion models for amortized inference.

Machine Learning (cs.LG); Machine Learning (stat.ML)

arXiv:2402.05098 [cs.LG]

(or arXiv:2402.05098v5 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2402.05098 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.52202/079017-2575 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.