analiza aiqbench
This audit reveals that numerical shifts in unlearning benchmarks for batch-normalized vision models stem from the checkpoint's inherent properties rather than the survival of removed data or methodological inconsistencies. The findings suggest that reported accuracy metrics are highly sensitive to fitting conventions, necessitating explicit documentation of these parameters alongside performance claims to ensure reproducibility. Consequently, the reliability of current unlearning evaluations depends on standardizing how batch normalization statistics are handled during refitting procedures.
Published Unlearning Numbers Move Per Checkpoint, and Not Because the Removed Data Survives: An Audit of 263 Released Batch-Normalized Checkpoints
arXiv:2609.11490v1 Announce Type: new Abstract: An unlearning audit reads its verdict off numbers that an unlearned model and its retrained reference each publish, and both also ship batch-normalization statistics that no gradient step wrote and no release records. Refitting them on kept data at bit-identical weights moves 47 of 221 released checkpoints past the spread their own release's seeds show, several inside a method whose average does not move: what moves is the checkpoint's property, not its method's. What does the moving is not the removed data surviving in the state: exchanging kept records for removed ones inside a fixed fitting pool moves a published cell by almost nothing, while how far a checkpoint's shipped state has drifted from any refit does track it. The consequence for a published decision is real but narrow: twelve verdicts cross, four clear a measured recalibration budget, two clear it on every replicate, and a population we trained and sited near its own criterion yields none. A release should therefore name the fitting convention beside the number, on the batch-normalized vision models where this channel exists.
Abstract:An unlearning audit reads its verdict off numbers that an unlearned model and its retrained reference each publish, and both also ship batch-normalization statistics that no gradient step wrote and no release records. Refitting them on kept data at bit-identical weights moves 47 of 221 released checkpoints past the spread their own release's seeds show, several inside a method whose average does not move: what moves is the checkpoint's property, not its method's. What does the moving is not the removed data surviving in the state: exchanging kept records for removed ones inside a fixed fitting pool moves a published cell by almost nothing, while how far a checkpoint's shipped state has drifted from any refit does track it. The consequence for a published decision is real but narrow: twelve verdicts cross, four clear a measured recalibration budget, two clear it on every replicate, and a population we trained and sited near its own criterion yields none. A release should therefore name the fitting convention beside the number, on the batch-normalized vision models where this channel exists.
38 pages, 4 figures, 26 tables. Independent of and concurrent with arXiv:2609.08901 (posted 8 Sep 2026): the instrument and protocol here were pre-registered on 29 Aug 2026; dated provenance in Appendix S
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.11490 [cs.AI]
(or arXiv:2609.11490v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.11490 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 230 cuvinte
Subiecte conexe
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · 3 aug. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · 4 sept. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026