When AI Generates Covariates: Causal Typing and Estimand Drift in Sequential Experiments

Abstract:AI-generated covariates from notes, conversations, images, and wearable streams can change the causal question when their roles are left unspecified. A generated feature may represent a treatment version, pre-action state, history, design variable, mediator, outcome proxy, observation process, or intercurrent event; these roles are not interchangeable. We formulate a causal type discipline for sequential experiments: a versioned representation map, a causal role classifier, a claim-status filter, and an estimand lock. The lock fixes a standardized proximal effect before generated covariates enter the analysis. Under audit correctness and standard identification assumptions, admissible role assignments preserve this estimand. We apply the established conditional-covariance characterization of compression bias to substitution of generated representations for design-relevant states. A standardized decomposition separates compression, conditional-law, and standardization drift. Further results cover mediator adjustment, post-action leakage, marker-intervention conflation, outcome-guided discovery, and state-measurement error. Cluster-level orthogonal estimators distinguish empirical and superpopulation targets under repeated sessions and missing outcomes. Simulations show that refinement helps when it retains design-relevant information, whereas design erasure, leakage, and same-data marker selection can produce bias or undercoverage. The framework places causal semantics and claim status before confirmatory inference with generated representations.

29 pages. Ancillary files include simulation code, seeds, and replicate-level results

Methodology (stat.ME); Artificial Intelligence (cs.AI)

arXiv:2609.17772 [stat.ME]

(or arXiv:2609.17772v1 [stat.ME] for this version)

https://doi.org/10.48550/arXiv.2609.17772 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

requirements.txt

results/table4_replicates.csv

results/table4_summary.csv

results/table5_replicates.csv

results/table5_summary.csv

results/table6_replicates.csv

results/table6_summary.csv

run_generated_covariate_simulations.py

run_remaining_parallel.py

verify_full_rerun.py

verify_results.py

(10 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.