Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Abstract:Importance-weighted variational inference approximates intractable densities by optimizing multi-sample Monte Carlo objectives whose tightness increases with the number of importance samples $N$. When reparameterization is unavailable, optimization relies on score-function (REINFORCE) gradient estimators. In this paper, we provide a comprehensive analysis of score-function gradient estimators for importance-weighted objectives by characterizing their asymptotic expectation and variance. We establish that standard variational inference for Monte Carlo objectives (VIMCO) gradient estimators suffer from a vanishing signal-to-noise ratio (SNR) as $N$ increases, which arises from a suboptimal control variate choice and can impair optimization. To address this issue, we derive a general mechanism for constructing VIMCO gradient estimators that avoid SNR collapse. We identify the High-Effective Sample Size optimal variance importance sampling estimator as belonging to this category of estimators and establish its $\sqrt{N}$ SNR scaling. This estimator does not, however, have zero variance at optimality and achieving its $\sqrt{N}$ SNR rate may require prohibitively large $N$ when the importance weights are degenerate. We overcome these limitations by introducing VIMCO gradient estimators that attain zero variance at optimality and support importance weight tempering. Experiments with discrete latent-variable and state-space models validate the theoretical predictions and illustrate the benefits of our estimators.

Machine Learning (stat.ML); Machine Learning (cs.LG)

arXiv:2602.01412 [stat.ML]

(or arXiv:2602.01412v2 [stat.ML] for this version)

https://doi.org/10.48550/arXiv.2602.01412 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.