aiqbench analysis
This research introduces PRDO, a novel framework for private decentralized learning that simultaneously mitigates sampling and privacy noise while correcting bias caused by heterogeneous data distributions. By establishing rigorous nonconvex convergence bounds without assuming uniform data heterogeneity, the study provides a theoretical foundation for more robust federated learning systems in privacy-sensitive environments. Experimental results confirm that this approach achieves superior accuracy compared to existing baselines, addressing critical challenges in distributed AI training.
Private Decentralized Optimization with Noise Reduction and Bias Correction
arXiv:2609.27658v1 Announce Type: new Abstract: Private decentralized learning is affected by sampling noise, privacy noise, and decentralized bias under heterogeneous data. We propose Private Recursive Decentralized Optimization (PRDO). PRDO uses recursive estimation with same-batch gradient differences to reduce estimation errors caused by sampling and privacy noise, while its Exact Diffusion component corrects decentralized bias arising from data heterogeneity. Our analysis establishes a nonconvex convergence bound without assuming uniformly bounded data heterogeneity across nodes. It further gives a sufficient condition under which recursive gradient differences yield strictly lower query sensitivity than private Exact Diffusion, together with an example that rigorously satisfies this condition. Experiments show improved accuracy over the evaluated baselines.
Abstract:Private decentralized learning is affected by sampling noise, privacy noise, and decentralized bias under heterogeneous data. We propose Private Recursive Decentralized Optimization (PRDO). PRDO uses recursive estimation with same-batch gradient differences to reduce estimation errors caused by sampling and privacy noise, while its Exact Diffusion component corrects decentralized bias arising from data heterogeneity. Our analysis establishes a nonconvex convergence bound without assuming uniformly bounded data heterogeneity across nodes. It further gives a sufficient condition under which recursive gradient differences yield strictly lower query sensitivity than private Exact Diffusion, together with an example that rigorously satisfies this condition. Experiments show improved accuracy over the evaluated baselines.
Machine Learning (cs.LG)
arXiv:2609.27658 [cs.LG]
(or arXiv:2609.27658v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.27658 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 199 words
