aiqbench analysis
This research addresses the granularity problem in machine unlearning by introducing a semantic mediation approach that surgically removes specific sensitive facts without discarding entire model responses. By generating counterfactual answers that retain benign explanations while excising targeted information, D2F significantly improves the balance between utility loss and privacy compliance compared to baseline methods. This advancement is critical for deploying large language models in regulated environments where precise knowledge removal is required.
Decouple to Forget: Semantic Mediation for Sensitive Unlearning in Large Language Models
arXiv:2610.17654v1 Announce Type: new Abstract: Machine unlearning for large language models seeks to remove specified knowledge while preserving general capability, yet current methods often treat an entire response as the unit of forgetting. This coarse granularity causes unnecessary utility loss when the forget target is only a small set of sensitive facts embedded within otherwise benign explanations. In this paper, we introduce Decouple to Forget (D2F), a targeted unlearning method inspired by semantic mediation that separates sensitive factual content from non-sensitive meaning and surface expression. For each example to be forgotten, D2F extracts a fact-level "mediator" representation from the answer, identifies which parts are sensitive, and removes only those parts while leaving the rest of the structure intact. We then generate counterfactual answers grounded in this mediator that preserve non-sensitive content but exclude sensitive facts, employing preference-based contrastive optimization to reduce the likelihood of information leakage while maintaining helpfulness and fluency. Experiments across various benchmark datasets show that D2F achieves a better balance between forgetting and retention than strong baseline methods. Our code is available at https://github.com/JingyueCong/D2F.
Abstract:Machine unlearning for large language models seeks to remove specified knowledge while preserving general capability, yet current methods often treat an entire response as the unit of forgetting. This coarse granularity causes unnecessary utility loss when the forget target is only a small set of sensitive facts embedded within otherwise benign explanations. In this paper, we introduce Decouple to Forget (D2F), a targeted unlearning method inspired by semantic mediation that separates sensitive factual content from non-sensitive meaning and surface expression. For each example to be forgotten, D2F extracts a fact-level "mediator" representation from the answer, identifies which parts are sensitive, and removes only those parts while leaving the rest of the structure intact. We then generate counterfactual answers grounded in this mediator that preserve non-sensitive content but exclude sensitive facts, employing preference-based contrastive optimization to reduce the likelihood of information leakage while maintaining helpfulness and fluency. Experiments across various benchmark datasets show that D2F achieves a better balance between forgetting and retention than strong baseline methods. Our code is available at this https URL.
Computation and Language (cs.CL)
arXiv:2610.17654 [cs.CL]
(or arXiv:2610.17654v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.17654 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words
Related coverage
Does Forgetting Transfer Across Modalities? A Real-World Benchmark for Cross-Modal Knowledge Unlearning Evaluation
arXiv · Aug 5, 2026
Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
arXiv · Aug 5, 2026
TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation
arXiv · Jun 9, 2026
Unlearning Is Not Just Erasing: Temporal Decoupling via Generation Inequality
arXiv · Aug 25, 2026