aiqbench analysis
This research introduces DPDL, a decentralized learning algorithm that mitigates privacy risks in non-IID data environments by applying differential privacy noise to cross-gradients and calibrating them via cosine similarity. Theoretically, the method establishes the minimum noise levels required for specific privacy guarantees while maintaining linear training speedup, effectively balancing model accuracy with robustness against privacy attacks. For practitioners in distributed systems, this approach offers a viable pathway to secure collaborative model training without relying on a central server.
DPDL: Towards Differential Privacy Preservation in Decentralized Stochastic Learning on Non-IID Data
arXiv:2606.04399v1 Announce Type: new Abstract: In the paradigm of decentralized learning, a group of agents collaborate to train a global model using distributed datasets without a central server. Although the power of collaboration has been verified by many state-of-the-art studies, it entails extensive gradient information exchanging among the agents and thus induces high risk of privacy leakage for the individual agents. Moreover, in real-world applications, the training data are usually non-identically and independently distributed across the agents, inducing more challenges to enable privacy-preserved decentralized learning. To address these issues, we propose a privacy-preserved decentralized learning algorithm with non-IID data, DPDL, which leverages the notion of Differential Privacy (DP) in cross-gradient aggregation through a similarity-based calibration technique. Specifically, in each round, each agent perturbs the cross-gradients (i.e., the derivatives of its neighbors' local model in its private local data) by Gaussian noise mechanism before sharing them with its neighbors; it then adopt cosine similarity to calibrate the received perturbed cross-gradients such that the aggregation of the calibrated cross-gradients can be utilized to effectively update local model in a momentum-like manner. Our rigorous theoretical analysis not only reveals the minimum noise level required to achieve a specific level of privacy preservation, but also illustrates that our algorithm still achieves a linear speedup in training with non-IID data. We finally conduct extensive experiments on real-world dataset to validate the effectiveness of our algorithm in defending privacy attacks and in training accurate models.
Abstract:In the paradigm of decentralized learning, a group of agents collaborate to train a global model using distributed datasets without a central server. Although the power of collaboration has been verified by many state-of-the-art studies, it entails extensive gradient information exchanging among the agents and thus induces high risk of privacy leakage for the individual agents. Moreover, in real-world applications, the training data are usually non-identically and independently distributed across the agents, inducing more challenges to enable privacy-preserved decentralized learning. To address these issues, we propose a privacy-preserved decentralized learning algorithm with non-IID data, DPDL, which leverages the notion of Differential Privacy (DP) in cross-gradient aggregation through a similarity-based calibration technique. Specifically, in each round, each agent perturbs the cross-gradients (i.e., the derivatives of its neighbors' local model in its private local data) by Gaussian noise mechanism before sharing them with its neighbors; it then adopt cosine similarity to calibrate the received perturbed cross-gradients such that the aggregation of the calibrated cross-gradients can be utilized to effectively update local model in a momentum-like manner. Our rigorous theoretical analysis not only reveals the minimum noise level required to achieve a specific level of privacy preservation, but also illustrates that our algorithm still achieves a linear speedup in training with non-IID data. We finally conduct extensive experiments on real-world dataset to validate the effectiveness of our algorithm in defending privacy attacks and in training accurate models.
Machine Learning (cs.LG); Cryptography and Security (cs.CR)
arXiv:2606.04399 [cs.LG]
(or arXiv:2606.04399v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.04399 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 332 words
Related coverage
Toward provably private learning from federated data
Google Research · Oct 2, 2026
Semantic Editing with Coupled Stochastic Differential Equations
arXiv · Jun 16, 2026
Descent-Guided Policy Gradient for Scalable Cooperative Multi-Agent Learning
arXiv · Sep 30, 2026
Privacy-Preserving Decentralized Federated Learning via Explainable Adaptive Differential Privacy
arXiv · Aug 18, 2026