aiqbench analysis
This research addresses a critical gap in Large Language Model safety by introducing AdaPop, which adaptively adjusts unlearning pressure based on data popularity rather than applying uniform gradients. The method significantly reduces information leakage of forgotten facts under paraphrased and adversarial queries while preserving knowledge retention for relevant topics. This advancement strengthens the reliability of LLMs in scenarios requiring dynamic memory management or privacy-preserving updates.
The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning
arXiv:2608.14229v1 Announce Type: new Abstract: Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.
Abstract:Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain balance via a dual-ascent controller that adjusts the retain penalty each epoch. Across three model families and two benchmarks, AdaPop leaks ~5x less forgotten content than competing methods under paraphrased queries and ~1.6x less under adversarial reformulations. We support our analysis with internal metrics: under our method, forget-set hidden states move further from the pre-unlearning model's states than under other methods, while retain-set representations remain close.
Computation and Language (cs.CL)
arXiv:2608.14229 [cs.CL]
(or arXiv:2608.14229v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.14229 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 219 words
Related coverage
Evaluating task vectors, unlearning and inoculation
LessWrong · Sep 20, 2026
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning
arXiv · Jun 8, 2026
New framework for auditing machine unlearning
Google Research · Jun 10, 2026