analiza aiqbench
This paper introduces FedSub, a novel algorithm that leverages subspace projection to enhance federated learning efficiency on heterogeneous data. By constraining local updates to low-dimensional subspaces and utilizing dual variables, it effectively mitigates client drift while significantly reducing communication and computational overhead. The work provides theoretical convergence analysis alongside experimental validation, offering a practical solution for scaling deep neural networks across diverse client environments.
An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
arXiv:2509.05213v2 Announce Type: replace Abstract: This work addresses the key challenges of applying federated learning to large-scale deep neural networks, particularly the issue of client drift due to data heterogeneity across clients and the high costs of communication, computation, and memory. We propose FedSub, an efficient subspace algorithm for federated learning on heterogeneous data. Specifically, FedSub utilizes subspace projection to guarantee local updates of each client within low-dimensional subspaces, thereby reducing communication, computation, and memory costs. Additionally, it incorporates low-dimensional dual variables to mitigate client drift. We provide convergence analysis that reveals the impact of key factors such as step size and subspace projection matrices on convergence. Experimental results demonstrate its efficiency.
Abstract:This work addresses the key challenges of applying federated learning to large-scale deep neural networks, particularly the issue of client drift due to data heterogeneity across clients and the high costs of communication, computation, and memory. We propose FedSub, an efficient subspace algorithm for federated learning on heterogeneous data. Specifically, FedSub utilizes subspace projection to guarantee local updates of each client within low-dimensional subspaces, thereby reducing communication, computation, and memory costs. Additionally, it incorporates low-dimensional dual variables to mitigate client drift. We provide convergence analysis that reveals the impact of key factors such as step size and subspace projection matrices on convergence. Experimental results demonstrate its efficiency.
Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
arXiv:2509.05213 [cs.LG]
(or arXiv:2509.05213v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2509.05213 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 205 cuvinte
Subiecte conexe
FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
arXiv · 10 iun. 2026
News
Quantum Computing Report · 25 iul. 2026
UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity
arXiv · 19 aug. 2026
Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning
arXiv · 24 iul. 2026