aiqbench analysis
This research introduces CreDRO, a novel framework that quantifies epistemic uncertainty by modeling disagreement arising from distribution shifts rather than just optimization randomness. By leveraging distributionally robust optimization to learn ensembles of plausible models, the method significantly improves out-of-distribution detection and selective classification in critical domains like healthcare. The approach represents a meaningful advancement in handling real-world data non-stationarity within machine learning systems.
Learning Credal Ensembles via Distributionally Robust Optimization
arXiv:2602.08470v3 Announce Type: replace Abstract: Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.
Abstract:Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2602.08470 [cs.LG]
(or arXiv:2602.08470v3 [cs.LG] for this version)
Full text under the publisher's open licence · 173 words
Related coverage
Chaotic quantum system learns data with simple controls
Quantum Zeitgeist · Oct 13, 2026
Compact set-valued deep ensembling in multi-class classification
arXiv · Oct 6, 2026
OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models
arXiv · Oct 1, 2026
Adversarial Purification by Consistency-aware Latent Space Optimization on Data Manifolds
arXiv · Aug 5, 2026