aiqbench analysis
This research exposes a critical flaw in standard federated conformal risk control where pooling calibration scores protects only the average institution while violating coverage guarantees at nearly 40% of individual sites. The authors propose "risk-curve shrinkage," a mechanism that interpolates between local and pooled calibration to ensure reliable performance across all participating hospitals without compromising patient-level validity.
When Average Calibration Fails: Site-Conditional Federated Conformal Risk Control
arXiv:2606.20115v3 Announce Type: replace Abstract: Conformal risk control (CRC) provides distribution-free segmentation guarantees by calibrating a prediction-set threshold on held-out data. In federated deployments, the standard approach pools calibration scores into a single threshold. We quantify, on real multi-institutional brain tumor data (FeTS-2022, 1,251 subjects, 20 institutions), a critical failure: naive pooled CRC protects the average hospital but violates coverage at 40% of individual institutions, with the worst site exceeding the target false-negative rate by 7.8 percentage points. We trace this failure to a hidden design choice: the aggregation weights implicitly determine whose coverage is protected. Sample-size weighting optimizes patient-level validity but can sacrifice institution-level reliability; equal-site weighting improves institution-level reliability on this benchmark at comparable efficiency, using only a single scalar per site. We propose risk-curve shrinkage as a principled mechanism: each site transmits its empirical risk curve (G scalars) and a single hyperparameter n0 smoothly interpolates between site-specific local calibration and sample-size-weighted pooled calibration. Leave-one-site-out sensitivity analysis identifies n0=19, achieving 2.7/20 violations at 2.0x stretch. Direct Lagrangian budget optimization fails by concentrating risk on vulnerable hospitals; the finite-sample correction term is essential: removing it triples violations. No patient-level images, masks, or per-volume scores leave any site.
Abstract:Conformal risk control (CRC) provides distribution-free segmentation guarantees by calibrating a prediction-set threshold on held-out data. In federated deployments, the standard approach pools calibration scores into a single threshold. We quantify, on real multi-institutional brain tumor data (FeTS-2022, 1,251 subjects, 20 institutions), a critical failure: naive pooled CRC protects the average hospital but violates coverage at 40% of individual institutions, with the worst site exceeding the target false-negative rate by 7.8 percentage points. We trace this failure to a hidden design choice: the aggregation weights implicitly determine whose coverage is protected. Sample-size weighting optimizes patient-level validity but can sacrifice institution-level reliability; equal-site weighting improves institution-level reliability on this benchmark at comparable efficiency, using only a single scalar per site. We propose risk-curve shrinkage as a principled mechanism: each site transmits its empirical risk curve (G scalars) and a single hyperparameter n0 smoothly interpolates between site-specific local calibration and sample-size-weighted pooled calibration. Leave-one-site-out sensitivity analysis identifies n0=19, achieving 2.7/20 violations at 2.0x stretch. Direct Lagrangian budget optimization fails by concentrating risk on vulnerable hospitals; the finite-sample correction term is essential: removing it triples violations. No patient-level images, masks, or per-volume scores leave any site.
10 pages, 4 figures, 2 tables. Submitted to the DeCaF Workshop at MICCAI 2026
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2606.20115 [cs.LG]
(or arXiv:2606.20115v3 [cs.LG] for this version)
Full text under the publisher's open licence · 224 words
Related coverage
[AINews] not much happened today
Latent Space · Oct 3, 2026
Federated Causal Inference from Multi-Site Observational Data via Propensity Score Aggregation
arXiv · Jun 15, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
Byzantine-Robust Federated RAG via Aligned Calibration and Fixed-Membership Conformal Prediction
arXiv · Sep 29, 2026