RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent

Abstract:Empirical risk minimization (ERM) is a cornerstone of modern machine learning. This paper focuses on the man-in-the-middle (MITM) attack, wherein adversaries exploit communication vulnerabilities between devices to inject malicious updates during training. To address this challenge, we propose RESIST (Resilient dEcentralized learning using conSensus gradIent deScenT), an optimization algorithm designed to be robust against adversarially compromised communication links. RESIST uses a multistep consensus gradient descent framework with robust-statistics-based screening of neighbor messages. It has a design parameter $J$ that controls the frequency of local gradient computation: each local gradient update is preceded by $J-1$ communication and robust aggregation rounds. Compared with methods that perform both communication and a local gradient update at every iteration, RESIST with $J>2$ uses fewer local gradient updates at the same communication budget. We establish geometric algorithmic convergence guarantees for strongly convex and Polyak-Lojasiewicz ERM problems, and sublinear and finite-horizon guarantees for smooth nonconvex ERM problems. For heterogeneous local objectives, these guarantees quantify neighborhoods for the relevant iterate, objective-value, and stationarity errors. In the strongly convex homogeneous case, convergence becomes exact. We also establish statistical learning-rate guarantees under common-population independent and identically distributed sampling assumptions, including regimes in which the statistical error vanishes as the sample size grows. Experimental results demonstrate the robustness of RESIST across diverse attack strategies, screening methods, and loss functions. The $J$-ablation experiments further show that larger $J$ can improve convergence and reduce the number of local gradient updates at a fixed communication budget.

99 pages, 14 figures, and 1 table; final version published in Transactions on Machine Learning Research (09/2026). Published version: this https URL

Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)

arXiv:2502.07977 [cs.LG]

(or arXiv:2502.07977v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2502.07977 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.