analiza aiqbench
FedSLoP addresses critical bottlenecks in federated learning by using stochastic low-rank gradient projections to significantly cut communication volume and client-side memory usage. Theoretical analysis confirms convergence to a first-order stationary point at a rate of $O(1/\sqrt{NT})$, while empirical results on heterogeneous MNIST datasets show competitive accuracy against standard baselines. This approach offers a principled optimization method for deploying machine learning models in resource-constrained environments without sacrificing performance.
FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
arXiv:2604.24012v3 Announce Type: replace Abstract: Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments. We introduce FedSLoP, a federated optimization algorithm that combines stochastic low-rank subspace projections of gradients, thereby reducing the dimension of communicated and stored updates while preserving optimization progress. On the theoretical side, we develop a detailed nonconvex convergence analysis under standard smoothness and bounded-variance assumptions, showing that FedSLoP is guaranteed to converge to a first-order stationary point at a rate of $O(1/\sqrt{NT})$. On the empirical side, we conduct extensive experiments on federated MNIST classification with heterogeneous data partitions, showing that FedSLoP substantially reduces communication volume and client-side memory while achieving competitive or better accuracy compared with FedAvg and representative sparse or low-rank baselines. Together, our results demonstrate that random subspace momentum methods such as FedSLoP provide a principled and effective approach to communication- and memory-efficient federated learning. Codes are available at: https://github.com/pkumelon/FedSLoP.git.
Abstract:Federated learning enables a population of clients to collaboratively train machine learning models without exchanging their raw data, but standard algorithms such as FedAvg suffer from slow convergence and high communication and memory costs in heterogeneous, resource-constrained environments. We introduce FedSLoP, a federated optimization algorithm that combines stochastic low-rank subspace projections of gradients, thereby reducing the dimension of communicated and stored updates while preserving optimization progress. On the theoretical side, we develop a detailed nonconvex convergence analysis under standard smoothness and bounded-variance assumptions, showing that FedSLoP is guaranteed to converge to a first-order stationary point at a rate of $O(1/\sqrt{NT})$. On the empirical side, we conduct extensive experiments on federated MNIST classification with heterogeneous data partitions, showing that FedSLoP substantially reduces communication volume and client-side memory while achieving competitive or better accuracy compared with FedAvg and representative sparse or low-rank baselines. Together, our results demonstrate that random subspace momentum methods such as FedSLoP provide a principled and effective approach to communication- and memory-efficient federated learning. Codes are available at: this https URL.
Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2604.24012 [cs.LG]
(or arXiv:2604.24012v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 187 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 3 oct. 2026
FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection
arXiv · 2 oct. 2026
CoCo-Fed: A Unified Framework for Memory- and Communication-Efficient Federated Learning at the Wireless Edge
arXiv · 28 iul. 2026
Accurate and Resource-Efficient Federated Continual Learning
arXiv · 12 iun. 2026