aiqbench analysis
This study empirically validates that natural experiments exist within real-world datasets and can be leveraged via causal feature selection to enhance model performance. By treating specific data subsets as interventional rather than purely observational, practitioners can potentially improve downstream accuracy in existing ML pipelines. The work serves as a foundational step toward systematically identifying and utilizing implicit interventions in large-scale data.
Do Real-World Datasets Contain Natural Experiments? An Empirical Study Using Causal Feature Selection
arXiv:2606.03251v1 Announce Type: new Abstract: In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic was an intervention by the coronavirus on the sub-population infected with COVID. We ask, do natural experiments occur in existing real-world datasets? If yes, how should we treat them? To detect natural experiments in data, we use causal discovery to recover the underlying causal graph and perform feature selection based on causal links. If downstream performance improves by treating the data as interventional rather than observational, we argue that this suggests the dataset contains natural experiments. We first validate this hypothesis by simulating datasets with and without natural experiments using synthetic graphs. We then perform a systematic empirical evaluation on a large suite of real-world datasets. Our results indicate that real-world datasets do contain natural experiments and we can take advantage of those natural experiments to improve model performance using causal inference. Our work represents the initial foray into this area, offering a preliminary exploration within a limited scope.
Abstract:In nature, events that affect some individuals or groups but not others constitute an implicit intervention and are known as natural experiments. For example, the COVID-19 pandemic was an intervention by the coronavirus on the sub-population infected with COVID. We ask, do natural experiments occur in existing real-world datasets? If yes, how should we treat them? To detect natural experiments in data, we use causal discovery to recover the underlying causal graph and perform feature selection based on causal links. If downstream performance improves by treating the data as interventional rather than observational, we argue that this suggests the dataset contains natural experiments. We first validate this hypothesis by simulating datasets with and without natural experiments using synthetic graphs. We then perform a systematic empirical evaluation on a large suite of real-world datasets. Our results indicate that real-world datasets do contain natural experiments and we can take advantage of those natural experiments to improve model performance using causal inference. Our work represents the initial foray into this area, offering a preliminary exploration within a limited scope.
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV); Machine Learning (stat.ML)
arXiv:2606.03251 [cs.AI]
(or arXiv:2606.03251v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.03251 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 211 words
Related coverage
Scaling-Score Conformal Prediction for Multi-Target Regression
arXiv · Sep 16, 2026
Mamba Hawkes Process for Event Sequence Modeling
arXiv · Oct 30, 2026
Learning to Zoom Efficiently with a Contrastive Curriculum
arXiv · Sep 4, 2026
A Framework for Evaluating and Benchmarking Concept Drift Detection Methods
arXiv · Jun 9, 2026