aiqbench analysis
This article introduces Causal Foundation Models (CFMs), a novel approach that adapts the foundation model paradigm to causal inference by enabling pretrained networks to estimate treatment effects via in-context learning without fine-tuning. The work bridges traditional bespoke causal pipelines with scalable ML architectures, offering practical code examples and notebooks for researchers exploring this emerging intersection of fields.
Causal Foundation Models
arXiv:2609.03003v1 Announce Type: new Abstract: Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.
Abstract:Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2609.03003 [cs.LG]
(or arXiv:2609.03003v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.03003 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 234 words
Related coverage
CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes
arXiv · Oct 1, 2026
Causal Analysis for Time Series Foundation Models
arXiv · Aug 26, 2026
DAG-FM: A Foundation Model for Causal Discovery under Heterogeneous Causal Mechanisms
arXiv · Aug 4, 2026
Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models
arXiv · Sep 17, 2026