analiza aiqbench
This research challenges the sufficiency of standard output-level metrics for machine unlearning, revealing that models can appear "forgotten" while retaining structured residuals inconsistent with true retraining. The study demonstrates that these hidden discrepancies create a vulnerability where information could potentially be recovered through relearning attacks, suggesting current evaluations overestimate actual forgetting success. Consequently, the field must shift toward assessing representation-level consistency rather than relying solely on accuracy drops or membership inference scores.
Erased, but Not Gone: Output Forgetting Is Not True Forgetting
arXiv:2606.25001v2 Announce Type: replace Abstract: Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced membership inference. But if output-level success can coexist with retraining-inconsistent residuals in representation space, what kind of forgetting are current evaluations actually certifying? We study this question through retraining-consistent representation forgetting, using a population of retrained models (i.e., trained from scratch without the forget data) as an operational reference distribution for ideal forgetting. Across multiple unlearning methods, datasets, and models, our theoretical analysis and empirical results show that standard output-level evaluation can systematically overestimate the success of unlearning. Under this stronger lens, current methods often appear forgotten at the output layer while exhibiting a structured mismatch relative to retraining. They partially align with retraining on forget samples, remain more inconsistent on retain samples, and tend to leave residual discrepancy localized along retraining-related directions rather than diffuse in representation space. This structured mismatch is characterized by forget/retain asymmetry, directional mismatch, and concentrated residuals along retraining-related directions. In many cases, the discrepancies exceed the variability of independent retraining and lead to a risk of information recovery during relearning attacks. Overall, the results suggest that current MU is often evaluated for apparent forgetting rather than retraining-consistent forgetting. More broadly, retraining reveals what output forgetting hides.
Abstract:Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced membership inference. But if output-level success can coexist with retraining-inconsistent residuals in representation space, what kind of forgetting are current evaluations actually certifying? We study this question through retraining-consistent representation forgetting, using a population of retrained models (i.e., trained from scratch without the forget data) as an operational reference distribution for ideal forgetting. Across multiple unlearning methods, datasets, and models, our theoretical analysis and empirical results show that standard output-level evaluation can systematically overestimate the success of unlearning. Under this stronger lens, current methods often appear forgotten at the output layer while exhibiting a structured mismatch relative to retraining. They partially align with retraining on forget samples, remain more inconsistent on retain samples, and tend to leave residual discrepancy localized along retraining-related directions rather than diffuse in representation space. This structured mismatch is characterized by forget/retain asymmetry, directional mismatch, and concentrated residuals along retraining-related directions. In many cases, the discrepancies exceed the variability of independent retraining and lead to a risk of information recovery during relearning attacks. Overall, the results suggest that current MU is often evaluated for apparent forgetting rather than retraining-consistent forgetting. More broadly, retraining reveals what output forgetting hides.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.25001 [cs.LG]
(or arXiv:2606.25001v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.25001 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 303 cuvinte
Subiecte conexe
Academia is for Ambition — Alex Zhang, MIT
Latent Space · 2 oct. 2026
Tariffs didn’t bring manufacturing jobs back to the US
The Verge · 27 iul. 2026
Microsoft’s AI chief says superintelligence is near, but won’t take your job
The Verge · 8 iun. 2026
Claude Code’s Next Era — Thariq Shihipar, Anthropic
Latent Space · 29 sept. 2026