aiqbench analysis
This research advances amortized Bayesian inference by comparing variational objectives for inverse problems, demonstrating that Wasserstein-based formulations significantly outperform standard KL-divergence approaches in ill-conditioned scenarios. The study provides a rigorous mathematical analysis using Fisher basis gradients to guide the selection of regularization strategies for complex physics-based models involving ODEs and PDEs. These findings are critical for improving posterior accuracy in scientific machine learning applications where parameter identifiability is challenging.
Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning
arXiv:2609.25145v1 Announce Type: cross Abstract: Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study investigates three objectives: a reverse Kullback--Leibler formulation (VAE-KL), an asymmetric Jensen--Shannon formulation (VAE-JS), and a Jensen--Shannon--Wasserstein formulation (VAE-JSWA), which replaces the reverse Kullback--Leibler regularizer with the squared 2-Wasserstein distance while retaining forward-Kullback--Leibler posterior supervision. A full-covariance Gaussian encoder and a pre-trained physics-based surrogate are used for amortized posterior inference. A local linear--Gaussian analysis in the generalized Fisher basis is developed to characterize the variance-dependent gradients of the three objectives. The formulations are first evaluated using linear--Gaussian benchmarks with known posterior solutions and subsequently tested on nonlinear physics-based inverse problems, including an inverse problem governed by a linear ODE and two PDE-constrained problems. VAE-KL performs slightly better than the other formulations in the well-conditioned benchmark, where all three approaches yield comparable posterior approximations, whereas VAE-JSWA provides substantially lower posterior errors in the strongly ill-conditioned benchmark. The nonlinear physics-based problems exhibit a similar conditioning-dependent trend, with JS-based formulations providing greater benefit as posterior ill-conditioning increases. These results indicate that posterior conditioning is an important factor in selecting variational objectives and motivate geometry-adaptive variational inference for Bayesian inverse problems.
Abstract:Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study investigates three objectives: a reverse Kullback--Leibler formulation (VAE-KL), an asymmetric Jensen--Shannon formulation (VAE-JS), and a Jensen--Shannon--Wasserstein formulation (VAE-JSWA), which replaces the reverse Kullback--Leibler regularizer with the squared 2-Wasserstein distance while retaining forward-Kullback--Leibler posterior supervision. A full-covariance Gaussian encoder and a pre-trained physics-based surrogate are used for amortized posterior inference. A local linear--Gaussian analysis in the generalized Fisher basis is developed to characterize the variance-dependent gradients of the three objectives. The formulations are first evaluated using linear--Gaussian benchmarks with known posterior solutions and subsequently tested on nonlinear physics-based inverse problems, including an inverse problem governed by a linear ODE and two PDE-constrained problems. VAE-KL performs slightly better than the other formulations in the well-conditioned benchmark, where all three approaches yield comparable posterior approximations, whereas VAE-JSWA provides substantially lower posterior errors in the strongly ill-conditioned benchmark. The nonlinear physics-based problems exhibit a similar conditioning-dependent trend, with JS-based formulations providing greater benefit as posterior ill-conditioning increases. These results indicate that posterior conditioning is an important factor in selecting variational objectives and motivate geometry-adaptive variational inference for Bayesian inverse problems.
Machine Learning (stat.ML); Computational Engineering, Finance, and Science (cs.CE); Information Theory (cs.IT); Machine Learning (cs.LG); Numerical Analysis (math.NA)
arXiv:2609.25145 [stat.ML]
(or arXiv:2609.25145v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2609.25145 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 247 words
Related coverage
Energy-based Transport for Amortized Bayesian Inference
arXiv · Jul 20, 2026
Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems
arXiv · Oct 5, 2026
Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design
arXiv · Aug 3, 2026
ProximalFM: Amortized Proximal Causal Inference under Hidden Confounding
arXiv · Oct 7, 2026