aiqbench analysis
This research advances conformal anomaly detection by introducing resampling-based methods that balance data efficiency with computational speed in low-data regimes. By controlling Type I error rates without inflating false negatives, these techniques offer practical statistical guarantees for building more trustworthy AI systems.
Leave-One-Out-, Bootstrap- and Cross-Conformal Anomaly Detectors
arXiv:2402.16388v4 Announce Type: replace-cross Abstract: The need for uncertainty quantification in anomaly detection systems has become increasingly important. In this context, effectively controlling Type I error rates without inflating Type II error rates in these systems can build trust and reduce costs associated with false discoveries. The field of conformal anomaly detection emerges as a promising approach for providing respective statistical and finite-sample validity guarantees through model calibration. However, reliance on calibration data imposes practical limitations, especially in low-data regimes. In this work, we formally define and evaluate leave-one-out-, bootstrap-, and cross-conformal methods for conformal anomaly detection, building on methods from the field of conformal prediction. Looking beyond the classical split-conformal approach, we show that derived methods for calculating resampling-conformal $p$-values offer a practical compromise between the data efficiency of full-conformal (transductive) approaches and the computational efficiency of split-conformal (inductive) methods. We validate derived methods and quantify their improvements for a range of one-class classifiers and datasets.
Abstract:The need for uncertainty quantification in anomaly detection systems has become increasingly important. In this context, effectively controlling Type I error rates without inflating Type II error rates in these systems can build trust and reduce costs associated with false discoveries. The field of conformal anomaly detection emerges as a promising approach for providing respective statistical and finite-sample validity guarantees through model calibration. However, reliance on calibration data imposes practical limitations, especially in low-data regimes. In this work, we formally define and evaluate leave-one-out-, bootstrap-, and cross-conformal methods for conformal anomaly detection, building on methods from the field of conformal prediction. Looking beyond the classical split-conformal approach, we show that derived methods for calculating resampling-conformal $p$-values offer a practical compromise between the data efficiency of full-conformal (transductive) approaches and the computational efficiency of split-conformal (inductive) methods. We validate derived methods and quantify their improvements for a range of one-class classifiers and datasets.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2402.16388 [stat.ML]
(or arXiv:2402.16388v4 [stat.ML] for this version)
Proc. 2024 IEEE ICKG 15(1): 110-119 (February 2025)
https://doi.org/10.1109/ICKG63256.2024.00022 DOI(s) linking to related resources
Full text under the publisher's open licence · 180 words
Related coverage
Conformal Anomaly Detection in Python: Moving Beyond Heuristic Thresholds with nonconform
arXiv · Jul 27, 2026
Machine learning for journal entry testing: A type-aware evaluation of anomaly detectors under a review budget
arXiv · Oct 7, 2026
Distribution-free false-alarm calibration and chance-corrected spatial evaluation for industrial anomaly detection
arXiv · Aug 19, 2026
Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searches
arXiv · Jun 15, 2026