analiza aiqbench
This research introduces SpecGradFilter, a novel framework that addresses statistical heterogeneity in Federated Learning by filtering low-frequency spectral components responsible for client drift. By leveraging frequency-domain analysis rather than traditional spatial corrections, the method achieves superior convergence in non-IID settings with minimal communication overhead. The findings establish a new paradigm for robust optimization in distributed machine learning environments.
SpecGradFilter: A Spectral Gradient Filtering Framework for Taming Federated Heterogeneity
arXiv:2607.04189v2 Announce Type: replace Abstract: Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.
Abstract:Federated Learning (FL) is fundamentally challenged by statistical heterogeneity, where non-identically distributed (non-IID) data induces client drift that severely hampers global convergence. While existing approaches attempt to mitigate this drift through spatial-domain gradient correction or regularization, they overlook the intrinsic spectral structure of optimization signals. In this work, we revisit client drift from a novel frequency-domain perspective and uncover a critical Spectral Bias of Drift: inter-client gradient divergence is predominantly concentrated in low-frequency components which encode client-specific distributional shifts, while high-frequency components representing fine-grained features remain relatively consistent. Motivated by this, we propose SpecGradFilter, a unified Spectral Gradient Filtering Framework that tames heterogeneity by suppressing discordant low-frequency signals. Crucially, we demonstrate that SpecGradFilter is a generalizable principle, effective not only via precise FFT-based truncation but also through spatial approximations like Gaussian detrending. Extensive experiments on benchmarks such as CIFAR-10/100 and Tiny-ImageNet demonstrate that SpecGradFilter significantly performs better performance in highly Non-IID settings with negligible communication overhead, establishing a new paradigm for robust federated optimization.
Machine Learning (cs.LG)
arXiv:2607.04189 [cs.LG]
(or arXiv:2607.04189v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.04189 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 256 cuvinte
Subiecte conexe
One Round Is All You Need: Analytic Federated Learning for Task-Heterogeneous Multi-Label Medical Image Classification
arXiv · 24 iul. 2026
Principal-timestep Restricted Init via Sparse Matrix-decomposition in Flow-matching
arXiv · 15 sept. 2026
RIPPLE in Still Water: Zero-Shot Clustering in Federated Learning with Wavelet Scattering Transform
arXiv · 5 oct. 2026
Robust Decentralized Federated Distillation via Multi-Modality Knowledge Collaboration
arXiv · 9 sept. 2026