analiza aiqbench
BLADE addresses critical robustness issues in large language model unlearning by introducing a constrained bilevel framework that stabilizes optimization landscapes during repeated forgetting steps. By utilizing clamped-entropy losses and asymmetric augmented Lagrangians, the method maintains model coherence even under significant scaling, outperforming existing baselines across multiple benchmark families. This advancement represents a meaningful step forward in making AI model modification more reliable and predictable for production environments.
BLADE: Bilevel Low-rank Augmented-Lagrangian Erasure for LLM Unlearning
arXiv:2608.22557v1 Announce Type: cross Abstract: Existing LLM unlearning methods struggle with robustness: unbounded forget losses degrade model coherence, fixed-weight balancing cannot adapt as retain difficulty shifts mid-training, and methods that work on one benchmark falter under scaling or repeated application. We propose BLADE, a constrained bilevel framework whose three mechanisms give smooth, predictable control over the optimization landscape: a clamped-entropy forget loss whose gradient is exactly zero once a token reaches sufficient uncertainty; an asymmetric augmented Lagrangian that permanently ratchets retain protection after any violation; and a bilevel structure confined to LoRA adapters that repairs retain damage before each forgetting step. BLADE dominates across three benchmark families, improving average composite scores over the strongest baselines by $6$% on TOFU, $9$% on MUSE Books, and $7$% on KnowUndo, and it remains stable under $4\times$ scaling and $4$ sequential unlearning steps on MUSE News where the best competing method collapses entirely.
Abstract:Existing LLM unlearning methods struggle with robustness: unbounded forget losses degrade model coherence, fixed-weight balancing cannot adapt as retain difficulty shifts mid-training, and methods that work on one benchmark falter under scaling or repeated application. We propose BLADE, a constrained bilevel framework whose three mechanisms give smooth, predictable control over the optimization landscape: a clamped-entropy forget loss whose gradient is exactly zero once a token reaches sufficient uncertainty; an asymmetric augmented Lagrangian that permanently ratchets retain protection after any violation; and a bilevel structure confined to LoRA adapters that repairs retain damage before each forgetting step. BLADE dominates across three benchmark families, improving average composite scores over the strongest baselines by $6$% on TOFU, $9$% on MUSE Books, and $7$% on KnowUndo, and it remains stable under $4\times$ scaling and $4$ sequential unlearning steps on MUSE News where the best competing method collapses entirely.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2608.22557 [cs.LG]
(or arXiv:2608.22557v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.22557 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 245 cuvinte
Subiecte conexe
SAUL: Sharpness-Aware Augmented-Lagrangian Unlearning
arXiv · 19 aug. 2026
Lethe: Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning
arXiv · 3 iun. 2026
Probe-Geometry Alignment: Erasing the Cross-Sequence Memorization Signature Below Chance
arXiv · 22 sept. 2026
GAMMA: Global Bit Allocation for Mixed-Precision Models under Arbitrary Budgets
arXiv · 28 aug. 2026