aiqbench analysis
This research establishes fundamental limits on "forget-only" machine unlearning, demonstrating that algorithms must sometimes retain information equivalent to the entire dataset to accurately simulate retraining without access to original data. The findings reveal a critical trade-off where standard training discards details necessary for future deletion requests, challenging current assumptions about model efficiency and privacy compliance.
Why Forget-Only Unlearning Needs Memorization
arXiv:2610.10519v1 Announce Type: new Abstract: Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method: different datasets can produce the same trained model but require very different outputs after the same examples are removed. Using this observation, we derive lower bounds on how accurately unlearning can match retraining and instantiate them for several standard learning algorithms. We then ask what must be true when forget-only unlearning succeeds. To this end, we derive lower bounds on what an algorithm must memorize about the training data to handle arbitrary deletion requests. For simple threshold learners, the required information can be as large as the entire dataset, even though ordinary training keeps only one boundary point. Overall, our results show that information discarded during ordinary learning may be needed later for deletion, so models designed for forget-only unlearning may need to retain more information than standard training does.
Abstract:Machine unlearning asks for a deletion algorithm whose output is close to retraining from scratch without the selected forget examples. In this work, we study forget-only unlearning, where the deletion algorithm receives only the trained model and the examples to forget, with no retained data or extra training information. We ask whether forget-only unlearning is always possible. We first show that this depends on the learning method: different datasets can produce the same trained model but require very different outputs after the same examples are removed. Using this observation, we derive lower bounds on how accurately unlearning can match retraining and instantiate them for several standard learning algorithms. We then ask what must be true when forget-only unlearning succeeds. To this end, we derive lower bounds on what an algorithm must memorize about the training data to handle arbitrary deletion requests. For simple threshold learners, the required information can be as large as the entire dataset, even though ordinary training keeps only one boundary point. Overall, our results show that information discarded during ordinary learning may be needed later for deletion, so models designed for forget-only unlearning may need to retain more information than standard training does.
Machine Learning (cs.LG); Information Theory (cs.IT); Machine Learning (stat.ML)
arXiv:2610.10519 [cs.LG]
(or arXiv:2610.10519v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.10519 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 297 words
Related coverage
Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation
arXiv · Aug 5, 2026
Models Designed to Forget: Machine Unlearning via Key Deletion
arXiv · Sep 30, 2026
UNBIND: UNlearning By INference-time Directional Steering for Code LLMs
arXiv · Sep 30, 2026
UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?
arXiv · Sep 30, 2026