aiqbench analysis
GRACE addresses a critical gap in machine unlearning by enabling the inference of forget and retain datasets from heterogeneous corpora when only limited examples of undesired behavior are available. By leveraging gradient-guided coreset selection, this method effectively removes specific behaviors while preserving overall model utility across various domains and architectures. It represents a significant advancement for deploying safe and compliant large language models in dynamic environments where training data boundaries are not pre-defined.
GRACE:Gradient-guided Coreset Selection for LLM Unlearning
arXiv:2608.28361v1 Announce Type: new Abstract: Machine Unlearning methods for Large Language Models typically assume pre-specified forget and retain sets. In realistic settings, however, requests may provide only a few examples of undesired behavior, requiring forget and retain sets to be inferred from heterogeneous corpora. We study this data-selection problem and propose GRACE , a gradient-guided coreset selection method that constructs both forget and retain sets for LLM unlearning. GRACE first computes a forget direction from seed examples that elicit the undesired behavior, then selects a compact forget coreset whose gradients approximate this direction using non-negative orthogonal matching pursuit. To preserve model utility, it selects retain examples after projecting out the forget direction and applying clustered orthogonal matching pursuit in the remaining gradient space. Across two target domains, two model families, and four unlearning algorithms, GRACE improves model utility while maintaining comparable forget quality, with particularly consistent gains over prior gradient-based selection methods.
Abstract:Machine Unlearning methods for Large Language Models typically assume pre-specified forget and retain sets. In realistic settings, however, requests may provide only a few examples of undesired behavior, requiring forget and retain sets to be inferred from heterogeneous corpora. We study this data-selection problem and propose GRACE , a gradient-guided coreset selection method that constructs both forget and retain sets for LLM unlearning. GRACE first computes a forget direction from seed examples that elicit the undesired behavior, then selects a compact forget coreset whose gradients approximate this direction using non-negative orthogonal matching pursuit. To preserve model utility, it selects retain examples after projecting out the forget direction and applying clustered orthogonal matching pursuit in the remaining gradient space. Across two target domains, two model families, and four unlearning algorithms, GRACE improves model utility while maintaining comparable forget quality, with particularly consistent gains over prior gradient-based selection methods.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2608.28361 [cs.AI]
(or arXiv:2608.28361v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2608.28361 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 244 words
Related coverage
GLOBE: Trajectory-Aligned Gradient Matching with Structured SparseOptimization for Coreset Selection
arXiv · Aug 5, 2026
Gradient Concentration, Not Weight Saliency, Explains Representation-Level Class Unlearning
arXiv · Jul 24, 2026
Efficient Coreset Selection via K-Nearest Neighbor Graphs
arXiv · Aug 19, 2026
Low-Budget Active Learning through Entropic Optimal Transport
arXiv · Oct 2, 2026