Online-Score-Aided Federated Learning for Resource-Constrained Wireless Clients with Continual Data Arrival

Abstract:Heterogeneous system configurations of distributed clients connected to the central server (CS) via a time-varying wireless network pose significant challenges for popular distributed machine learning (ML) algorithms such as federated learning (FL). Although the limited (radio and computational) resources are widely acknowledged, two critical yet often ignored aspects are (a) client devices can only dedicate a small chunk of their limited storage for the FL task and (b) new training samples may arrive continually in many practical wireless applications. Therefore, we propose a new FL algorithm, online-score-aided federated learning (OSAFL), specifically designed for tasks with continual data arrival in resource-constrained environments. We first theoretically show how the convergence bound is affected by continual data distribution shifts, uncertain client participation, gradient quantization errors, and noise from stochastic gradients and statistical data heterogeneity across clients. We then show how to (sub-optimally) minimize these errors by choosing appropriate aggregation weights at the CS during global update. Our extensive simulation results across three popular image classification datasets and three ML models with different numbers of trainable parameters validate the effectiveness of the proposed OSAFL algorithm compared to (modified) state-of-the art FL baselines.

Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI); Systems and Control (eess.SY)

arXiv:2408.05886 [cs.LG]

(or arXiv:2408.05886v5 [cs.LG] for this version)