analiza aiqbench
This research introduces Robust Fed-Q, a federated reinforcement learning algorithm designed to achieve convergence even when a fraction of agents behave adversarially. By integrating model-based and model-free techniques with robust statistical methods, the approach minimizes communication overhead to nearly constant rounds while maintaining near-optimal sample efficiency. The work addresses critical scalability and security challenges in collaborative machine learning systems where bandwidth is a primary constraint.
Robust Federated Q-Learning with Almost No Communication
arXiv:2609.20174v1 Announce Type: new Abstract: We consider a federated reinforcement learning setting involving $M$ agents, all of whom interact with a common Markov Decision Process (MDP). The agents exchange information via a central server to learn the optimal value function. Our goal is to understand to what extent one can hope for collaborative sample-complexity speedups in such a setting, when a small fraction of the agents are adversarial and can act arbitrarily. To that end, we propose Robust Fed-Q}, a federated Q-learning algorithm that blends ideas from both model-based and model-free RL, along with the median-of-means device from robust statistics. We prove that despite corruption, with high-probability, Robust Fed-Q (i) guarantees exact convergence to the optimal value function in the limit of infinite samples, and (ii) enjoys near-optimal finite-time rates that benefit from collaboration. In addition, our approach requires just $\tilde{O}(1)$ rounds of communication to achieve each of the above guarantees, a feature of independent interest in FL where communication is the major bottleneck.
Abstract:We consider a federated reinforcement learning setting involving $M$ agents, all of whom interact with a common Markov Decision Process (MDP). The agents exchange information via a central server to learn the optimal value function. Our goal is to understand to what extent one can hope for collaborative sample-complexity speedups in such a setting, when a small fraction of the agents are adversarial and can act arbitrarily. To that end, we propose Robust Fed-Q}, a federated Q-learning algorithm that blends ideas from both model-based and model-free RL, along with the median-of-means device from robust statistics. We prove that despite corruption, with high-probability, Robust Fed-Q (i) guarantees exact convergence to the optimal value function in the limit of infinite samples, and (ii) enjoys near-optimal finite-time rates that benefit from collaboration. In addition, our approach requires just $\tilde{O}(1)$ rounds of communication to achieve each of the above guarantees, a feature of independent interest in FL where communication is the major bottleneck.
Machine Learning (cs.LG); Systems and Control (eess.SY)
arXiv:2609.20174 [cs.LG]
(or arXiv:2609.20174v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.20174 arXiv-issued DOI via DataCite (pending registration)
S. Maity and A. Mitra, "Robust Federated Q-Learning with Almost No Communication," 2026 American Control Conference (ACC), New Orleans, LA, USA, 2026, pp. 462-469
Text integral sub licența deschisă a publicației · 205 cuvinte
