aiqbench analysis
This paper introduces Rashomon Partition Sets (RPS), a Bayesian framework that enumerates high-evidence models to robustly estimate heterogeneity in factorial data without relying on sampling. By utilizing an l0 prior, the approach captures complex outcome variations while maintaining interpretability and offering minimax optimality from an information-theoretic perspective. The method provides rigorous bounds on approximation errors and the size of the model set, addressing the trade-off between model simplicity and the risk of spurious patterns in observational studies and randomized trials.
Robustly estimating heterogeneity in factorial data using Rashomon Partitions
arXiv:2404.02141v5 Announce Type: replace-cross Abstract: In both observational data and randomized control trials, researchers select statistical models to articulate how the outcome of interest varies with combinations of observable covariates. Choosing a model that is too simple can obfuscate important heterogeneity in outcomes between covariate groups, while too much complexity risks identifying spurious patterns. In this paper, we propose a novel Bayesian framework for model uncertainty called Rashomon Partition Sets (RPSs). The RPS consists of all models that have posterior density close to the maximum a posteriori (MAP) model. We construct the RPS by enumeration, rather than sampling, which ensures that we explore all models with high evidence in the data, even if they offer dramatically different substantive explanations. We use a l0 prior, which allows the allows us to capture complex heterogeneity without imposing strong assumptions about the associations between effects, showing this prior is minimax optimal from an information-theoretic perspective. We characterize the approximation error of (functions of) parameters computed conditional on being in the RPS relative to the entire posterior. We propose an algorithm to enumerate the RPS from the class of models that are interpretable and unique, then provide bounds on the size of the RPS. We give simulation evidence along with three empirical examples: price effects on charitable giving, heterogeneity in chromosomal structure, and the introduction of microfinance.
Abstract:In both observational data and randomized control trials, researchers select statistical models to articulate how the outcome of interest varies with combinations of observable covariates. Choosing a model that is too simple can obfuscate important heterogeneity in outcomes between covariate groups, while too much complexity risks identifying spurious patterns. In this paper, we propose a novel Bayesian framework for model uncertainty called Rashomon Partition Sets (RPSs). The RPS consists of all models that have posterior density close to the maximum a posteriori (MAP) model. We construct the RPS by enumeration, rather than sampling, which ensures that we explore all models with high evidence in the data, even if they offer dramatically different substantive explanations. We use a l0 prior, which allows the allows us to capture complex heterogeneity without imposing strong assumptions about the associations between effects, showing this prior is minimax optimal from an information-theoretic perspective. We characterize the approximation error of (functions of) parameters computed conditional on being in the RPS relative to the entire posterior. We propose an algorithm to enumerate the RPS from the class of models that are interpretable and unique, then provide bounds on the size of the RPS. We give simulation evidence along with three empirical examples: price effects on charitable giving, heterogeneity in chromosomal structure, and the introduction of microfinance.
Methodology (stat.ME); Machine Learning (cs.LG); Econometrics (econ.EM); Computation (stat.CO); Machine Learning (stat.ML)
arXiv:2404.02141 [stat.ME]
(or arXiv:2404.02141v5 [stat.ME] for this version)
Full text under the publisher's open licence · 238 words
Related coverage
Uncertainty-Guided LLM Semantic Augmentation for Heterogeneous Treatment Effect Estimation
arXiv · Jul 30, 2026
Probabilistic Multi-Robot Gas Source Localization with Uncalibrated Sensors: A Distributed Estimation Approach
arXiv · Aug 31, 2026
Capacity Confounds and Coverage Guarantees in Adaptive Sub-model Federated Learning
arXiv · Oct 9, 2026
Orthogonal Learner for Estimating Heterogeneous Long-Term Treatment Effects
arXiv · Jun 5, 2026