aiqbench analysis
Lethe addresses a critical gap in federated learning by preventing "knowledge resurfacing," where previously erased data re-emerges during continued model training. By decoupling forget and retain streams through a temporary adapter mechanism, the method ensures persistent unlearning with a resurfacing rate below 1% even after extensive follow-up iterations. This approach offers a robust solution for maintaining data privacy guarantees in dynamic, long-term federated deployments.
Lethe: Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning
arXiv:2601.22601v2 Announce Type: replace Abstract: Federated unlearning (FU) aims to erase designated client-level, class-level, or sample-level knowledge from a global model. Existing studies commonly assume that the collaboration ends with the unlearning operation, overlooking the follow-up situation where federated training continues over the remaining data. We identify a critical failure mode, termed knowledge resurfacing, by revealing that continued training can re-activate unlearned knowledge and cause the removed influence to resurface in the global model. To address this, we propose Lethe, a novel federated unlearning method that de-correlates knowledge to be unlearned from knowledge to be retained, ensuring persistent erasure during continued training. Lethe follows a Reshape--Rectify--Restore pipeline: a temporary adapter is first trained with gradient ascent on the unlearning data to obtain magnified updates, which are then used as corrective signals to guide layer-wise rectification of the remaining updates in two streams. Finally, the adapter is removed, and a short recovery stage is performed on the retained data. Our experiments show that Lethe supports unlearning at all levels in federated systems in a unified manner and maintains superior persistence, with a resurfacing rate below 1% in most cases, even after numerous rounds of follow-up training.
Abstract:Federated unlearning (FU) aims to erase knowledge from a global model. Existing studies commonly assume that federated collaboration terminates after unlearning, overlooking a deployment-realistic scenario where training continues on the remaining clients after deletion requests are fulfilled. In this work, we identify a critical failure mode, termed knowledge resurfacing, revealing that continued training on retained data alone can reactivate unlearned knowledge in a few rounds. Empirically, we demonstrate that many state-of-the-art FU methods are prone to knowledge resurfacing. We then propose Lethe, a novel unlearning method for persistent knowledge erasure in federated settings. In each iteration, Lethe operates on a forget stream from the unlearning client and a retain stream from the retained clients. It redirects unlearning updates toward a region where the two streams are anti-aligned, discouraging retained-data training from moving back toward the forgotten knowledge. Consequently, Lethe ensures stronger unlearning persistence during subsequent federated training. Extensive experiments across diverse models, datasets, and unlearning levels validate that Lethe supports all levels of unlearning in a unified manner across both CV and NLP tasks, demonstrating consistently low RR, below 1% in most cases, even after an extremely long horizon of follow-up training.
Machine Learning (cs.LG)
arXiv:2601.22601 [cs.LG]
(or arXiv:2601.22601v3 [cs.LG] for this version)
Full text under the publisher's open licence · 203 words
Related coverage
Democratizing institutional knowledge: Building an AI-powered knowledge management system with AWS
AWS · Aug 24, 2026
Modernizing and scaling support operations with generative AI on AWS
AWS · Sep 2, 2026
Lethe: Principled Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning
arXiv · Oct 6, 2026
\textsc{Lethe}: Principled Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning
arXiv · Jun 5, 2026