aiqbench analysis
This paper introduces a critical new benchmark for evaluating causal inference methods in epidemic time series by generating realistic counterfactual trajectories using agent-based models. It addresses the significant gap where existing datasets lack ground-truth outcomes or fail to capture complex, dynamic intervention scenarios like multi-policy settings. By enabling rigorous testing of deep learning approaches under static and time-varying treatments, this work sets a new standard for validating causal reasoning capabilities in AI systems handling sequential data.
Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions
arXiv:2606.05692v3 Announce Type: replace Abstract: Deep learning has enabled significant advances in time-series causal inference, yet progress remains constrained by the lack of realistic benchmarks with observable counterfactual outcomes. Existing datasets either rely on real-world observations without ground-truth counterfactuals or on simplified simulations that fail to capture complex causal dynamics. To address this gap, we develop a large-scale benchmark for counterfactual prediction in epidemic time series under dynamic interventions. Unlike existing benchmarks, it supports static and time-varying treatments, as well as both single-policy and multi-policy intervention settings, enabling evaluation of causal inference methods across a broad range of causal inference scenarios. Leveraging a calibrated agent-based model grounded in real-world demographic, mobility, epidemiological, and policy data, we generate realistic counterfactual trajectories across more than 150 U.S. counties. Using this benchmark, we evaluate widely used and state-of-the-art causal inference methods, revealing substantial performance differences and highlighting the challenges of realistic time-series causal reasoning.
Abstract:Deep learning has enabled significant advances in time-series causal inference, yet progress remains constrained by the lack of realistic benchmarks with observable counterfactual outcomes. Existing datasets either rely on real-world observations without ground-truth counterfactuals or on simplified simulations that fail to capture complex causal dynamics. To address this gap, we develop a large-scale benchmark for counterfactual prediction in epidemic time series under dynamic interventions. Unlike existing benchmarks, it supports static and time-varying treatments, as well as both single-policy and multi-policy intervention settings, enabling evaluation of causal inference methods across a broad range of causal inference scenarios. Leveraging a calibrated agent-based model grounded in real-world demographic, mobility, epidemiological, and policy data, we generate realistic counterfactual trajectories across more than 150 U.S. counties. Using this benchmark, we evaluate widely used and state-of-the-art causal inference methods, revealing substantial performance differences and highlighting the challenges of realistic time-series causal reasoning.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.05692 [cs.LG]
(or arXiv:2606.05692v3 [cs.LG] for this version)
https://doi.org/10.1145/3770855.3817522 DOI(s) linking to related resources
Full text under the publisher's open licence · 166 words
Related coverage
Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions
arXiv · Jun 12, 2026
Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventions
arXiv · Jun 6, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Causal Longitudinal Prior-Fitted Networks for Counterfactual Outcome Prediction
arXiv · Jun 9, 2026