aiqbench analysis
This research introduces MUNKEY, a memory-augmented transformer architecture that enables efficient machine unlearning by design rather than relying on computationally expensive post-hoc parameter updates. By decoupling instance-specific memorization from core weights through key deletion, the method allows for zero-shot forgetting without access to original training data, addressing critical privacy and safety constraints in deployed vision models. The approach demonstrates superior performance across natural image and medical benchmarks compared to existing baselines, offering a viable path toward scalable compliance with evolving data governance regulations.
Models Designed to Forget: Machine Unlearning via Key Deletion
arXiv:2603.15033v4 Announce Type: replace Abstract: Machine unlearning for vision models is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training images. Despite this, most existing approximate unlearning methods tackle the problem from a post-hoc perspective. They attempt to erase the influence of targeted samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We motivate unlearning by design, a novel paradigm for approximate methods in which models are directly trained to support forgetting as an inherent architectural capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory-augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained visual recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.
Abstract:Machine unlearning for vision models is rapidly becoming a practical requirement, driven by privacy regulations, data errors, and the need to remove harmful or corrupted training images. Despite this, most existing approximate unlearning methods tackle the problem from a post-hoc perspective. They attempt to erase the influence of targeted samples through parameter updates that typically require access to the full training data. This creates a mismatch with real deployment scenarios where unlearning requests can be anticipated, revealing a fundamental limitation of post-hoc approaches. We motivate unlearning by design, a novel paradigm for approximate methods in which models are directly trained to support forgetting as an inherent architectural capability. We instantiate this idea with Machine UNlearning via KEY deletion (MUNKEY), a memory-augmented transformer that decouples instance-specific memorization from model weights. Here, unlearning corresponds to removing the instance-identifying key, enabling zero-shot forgetting without weight updates or access to the original samples or labels. Across natural image benchmarks, fine-grained visual recognition, and medical datasets, MUNKEY outperforms all post-hoc baselines. Our results establish that unlearning by design enables fast, deployment-oriented unlearning while preserving predictive performance.
Machine Learning (cs.LG)
arXiv:2603.15033 [cs.LG]
(or arXiv:2603.15033v4 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2603.15033 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
Why Forget-Only Unlearning Needs Memorization
arXiv · Oct 8, 2026
CUNO: Curriculum and Preference Optimization for Stable Graph Unlearning under Mass Deletion
arXiv · Sep 9, 2026
New framework for auditing machine unlearning
Google Research · Jun 10, 2026
Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions
arXiv · Jun 9, 2026