analiza aiqbench
This paper treats federated learning on drifting data as an optimization problem: each client gets its own history window sized to its noise, arrival rate, and drift, instead of one shared window. The authors give a tracking-error bound and a matching lower bound, and their experiments attribute the gain to the per-client memory rather than the solver. For ML practitioners, the takeaway is that a single global memory setting can be provably suboptimal when clients are heterogeneous, though the results are theoretical and the practical benefit depends on how well those client parameters can be estimated in deployment.
Beyond Aggregate Drift: Optimal Client-Specific Windows in Federated Learning
arXiv:2610.15543v1 Announce Type: cross Abstract: We study federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss. Clients are not alike as they collect data at different rates, with different noise, and change at different speeds. The main question is how much history should each one keep? A single shared window must serve every client at once, and so makes a fast-changing client carry the sampling noise of a slow, nearly stationary one. We give each client a memory of its own, sized to that client's noise, arrival rate, and drift, and refresh it by restarted accelerated method that starts from a common point. The tracking error is then bounded by \[ R\left(\sum_m w_m^{3/2}\,\sigma_m\sqrt{\gamma_m/\lambda_m}\right)^{2/3}, \] for clients with weight $w_m$, noise $\sigma_m$, arrival rate $\lambda_m$, and drift rate $\gamma_m$ on a domain of diameter $R$. This is never worse than what any common window can achieve, and strictly better whenever noise and drift are spread unevenly across clients. A lower bound that puts different groups of clients on different drift clocks shows this quantity is the right one. The two bounds agree up to a constant whenever the clients occupy boundedly many refresh scales, and in general differ only by the sixth root of the number of scales. Experiments show the gain comes from the memory rather than from the solver: it persists across accelerated, stochastic-gradient, and one-step methods, and against a separately tuned common-memory baseline.
Abstract:We study federated learning from continuously arriving data: each client samples from its own distribution, these distributions drift over time, and the server must track the minimizer of the clients' weighted average loss. Clients are not alike as they collect data at different rates, with different noise, and change at different speeds. The main question is how much history should each one keep? A single shared window must serve every client at once, and so makes a fast-changing client carry the sampling noise of a slow, nearly stationary one. We give each client a memory of its own, sized to that client's noise, arrival rate, and drift, and refresh it by restarted accelerated method that starts from a common point. The tracking error is then bounded by \[ R\left(\sum_m w_m^{3/2}\,\sigma_m\sqrt{\gamma_m/\lambda_m}\right)^{2/3}, \] for clients with weight $w_m$, noise $\sigma_m$, arrival rate $\lambda_m$, and drift rate $\gamma_m$ on a domain of diameter $R$. This is never worse than what any common window can achieve, and strictly better whenever noise and drift are spread unevenly across clients. A lower bound that puts different groups of clients on different drift clocks shows this quantity is the right one. The two bounds agree up to a constant whenever the clients occupy boundedly many refresh scales, and in general differ only by the sixth root of the number of scales. Experiments show the gain comes from the memory rather than from the solver: it persists across accelerated, stochastic-gradient, and one-step methods, and against a separately tuned common-memory baseline.
Optimization and Control (math.OC); Machine Learning (cs.LG)
arXiv:2610.15543 [math.OC]
(or arXiv:2610.15543v1 [math.OC] for this version)
https://doi.org/10.48550/arXiv.2610.15543 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 350 cuvinte
Subiecte conexe
Article: Governing AI in the Cloud: A Practical Guide for Architects
InfoQ · 15 iun. 2026
Quantum Machine Learning Overcomes Hardware Flaws with New Correction Method
Quantum Zeitgeist · 2 iun. 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · 8 mai 2026
AlignFed: Alignment-Aware Asynchronous Federated Fine-Tuning for Large Language Models in Heterogeneous Edge Environments
arXiv · 9 iun. 2026