aiqbench analysis
This paper proposes a transformer-based conditional VAE that amortizes causal knowledge across many training tasks, so it can predict interventional distributions from observational data in a single forward pass without a causal graph or interventional data. The evidence is limited to synthetic and gene-expression simulations, plus one finite-capacity test of the theory, and only a parametric simulator shift was tested out of distribution. Readers should weigh whether the amortization generalizes beyond these settings.
ACTIVA: Amortized Causal Effect Estimation via Variational Autoencoders
arXiv:2503.01290v4 Announce Type: replace Abstract: Predicting post-intervention distributions from observational data is central to many scientific and decision-making problems, but remains challenging due to causal ambiguity, restrictive modeling assumptions, and the lack of amortization across tasks. We introduce ACTIVA, a transformer-based conditional variational autoencoder for amortized estimation of full interventional distributions from observational data and intervention queries. ACTIVA learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, so that a new causal model drawn from the training distribution is handled in a single forward pass, without interventional data, a causal graph, or per-task fitting. Beyond the training distribution, only a parametric shift of the simulator is tested. We provide a population-level characterization of ACTIVA's estimand, showing that, under idealized expressivity and infinite samples assumptions, ACTIVA's learning objective targets a simulator-weighted mixture over the interventional distributions of causal models that are observationally compatible with the input. Empirically, on synthetic datasets and biologically realistic gene-expression simulations, ACTIVA substantially outperforms a correlational baseline, reduces spurious non-descendant effects, and achieves competitive performance relative to strong amortized baselines. On a pair of observationally equivalent models sharing a single observational dataset, ACTIVA recovers the predicted mixture, validating the theory in a finite-capacity model. Our results show that ACTIVA is a promising approach for estimating interventional distributions from observational data.
Abstract:Predicting post-intervention distributions from observational data is central to many scientific and decision-making problems, but remains challenging due to causal ambiguity, restrictive modeling assumptions, and the lack of amortization across tasks. We introduce ACTIVA, a transformer-based conditional variational autoencoder for amortized estimation of full interventional distributions from observational data and intervention queries. ACTIVA learns a conditional latent prior that amortizes causal knowledge across diverse training tasks, so that a new causal model drawn from the training distribution is handled in a single forward pass, without interventional data, a causal graph, or per-task fitting. Beyond the training distribution, only a parametric shift of the simulator is tested. We provide a population-level characterization of ACTIVA's estimand, showing that, under idealized expressivity and infinite samples assumptions, ACTIVA's learning objective targets a simulator-weighted mixture over the interventional distributions of causal models that are observationally compatible with the input. Empirically, on synthetic datasets and biologically realistic gene-expression simulations, ACTIVA substantially outperforms a correlational baseline, reduces spurious non-descendant effects, and achieves competitive performance relative to strong amortized baselines. On a pair of observationally equivalent models sharing a single observational dataset, ACTIVA recovers the predicted mixture, validating the theory in a finite-capacity model. Our results show that ACTIVA is a promising approach for estimating interventional distributions from observational data.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2503.01290 [cs.LG]
(or arXiv:2503.01290v4 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2503.01290 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 307 words
Related coverage
Kankeu and Colleagues Present Hybrid Variational Autoencoder for Neural Topic Modeling
Quantum Zeitgeist · Jun 17, 2026
Target-Aware State-Adaptive $p$-Dirichlet Graph Neural Regression for Non-Invasive Body-Composition Estimation
arXiv · Sep 1, 2026
Variational decomposition autoencoding improves disentanglement of latent representations
arXiv · Nov 2, 2026
Generalized Graph Variational Autoencoders: Bounded Divergences Control Posterior Collapse
arXiv · Sep 26, 2026