aiqbench analysis
This research introduces ADU, a novel framework that refines model unlearning by decoupling specific attention pathways rather than erasing tokens, thereby better preserving general utility while removing sensitive data. By targeting the retrieval paths of persistent anchors within local and global attention heads, the method achieves superior forget quality on standard benchmarks with minimal side effects in benign contexts. This advancement addresses critical privacy and safety constraints for deploying large language models in regulated environments.
Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality
arXiv:2608.23020v1 Announce Type: cross Abstract: Large language models (LLMs) require effective unlearning to address privacy regulations and safety concerns. However, achieving precise forgetting without compromising general utility remains challenging. Existing sequence- and token-level methods penalize target outputs without modeling their context-dependent retrieval paths, which can disrupt linguistic structure or suppress benign knowledge. We present ADU, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling. Exploiting the functional distinction between local and global attention heads, ADU identifies preplan positions that retrieve persistent sensitive anchors and fixes their candidate paths under the original model. It then trains attention-projection adapters to suppress attention mass along these paths while preserving local-attention structure and retain-set language modeling. Post-training activation exchange tests whether the modified attention-output module transmits the learned forgetting effect. ADU achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks, including a Forget Quality of (0.93) on TOFU. It preserves 87--98% of model utility (92.9% on average versus 81.9% for baselines) while reducing side effects in benign contexts.
Abstract:Large language models (LLMs) require effective unlearning to address privacy regulations and safety concerns. However, achieving precise forgetting without compromising general utility remains challenging. Existing sequence- and token-level methods penalize target outputs without modeling their context-dependent retrieval paths, which can disrupt linguistic structure or suppress benign knowledge. We present ADU, a fine-grained, training-based framework that shifts unlearning from token erasure to contextual attention-pathway decoupling. Exploiting the functional distinction between local and global attention heads, ADU identifies preplan positions that retrieve persistent sensitive anchors and fixes their candidate paths under the original model. It then trains attention-projection adapters to suppress attention mass along these paths while preserving local-attention structure and retain-set language modeling. Post-training activation exchange tests whether the modified attention-output module transmits the learned forgetting effect. ADU achieves the strongest aggregate performance among evaluated baselines on the TOFU and WMDP benchmarks, including a Forget Quality of (0.93) on TOFU. It preserves 87--98% of model utility (92.9% on average versus 81.9% for baselines) while reducing side effects in benign contexts.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2608.23020 [cs.CL]
(or arXiv:2608.23020v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.23020 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 267 words
Related coverage
Decoupling Knowledge Forgetting and Refusal Learning in MLLM Unlearning
arXiv · Oct 15, 2026
Decoupled Temporal Encoding for Generative Recommendation
arXiv · Aug 18, 2026
Compositional Concepts Erasing: Disentangling Object-Style via Domain Manifold Unlearning
arXiv · Oct 15, 2026
Unlearning on Spatio-Temporal Graphs through Subgraph Virtual Edge Reconstruction
arXiv · Sep 1, 2026