aiqbench analysis
This paper introduces DP-{\lambda}CGD, an algorithm that enhances the accuracy of differentially private model training by optimizing noise correlation without incurring significant memory overhead. By regenerating previous noise vectors instead of storing them, the method maintains a memory footprint comparable to standard DP-SGD while reducing computational costs. These improvements directly address key efficiency bottlenecks in deploying privacy-preserving machine learning systems at scale.
DP-{\lambda}CGD: Efficient Noise Correlation for Differentially Private Model Training
arXiv:2601.22334v3 Announce Type: replace Abstract: Differentially private stochastic gradient descent (DP-SGD) is the gold standard for training machine learning models with formal differential privacy guarantees. Several recent extensions improve its accuracy by introducing correlated noise across training iterations. Matrix factorization mechanisms are a prominent example, but they can require storing previously added noise vectors, leading to substantial memory overhead. In this work, we propose a new algorithm for correlated-noise private training that correlates noise only with the immediately preceding iteration and cancels a tunable portion of it. By regenerating the previous noise using a pseudorandom number generator rather than storing it, our method has essentially the same memory footprint as DP-SGD. We show that the computational overhead is minimal and empirically demonstrate improved accuracy over DP-SGD.
Abstract:Differentially private stochastic gradient descent (DP-SGD) is the gold standard for training machine learning models with formal differential privacy guarantees. Several recent extensions improve its accuracy by introducing correlated noise across training iterations. Matrix factorization mechanisms are a prominent example, but they can require storing previously added noise vectors, leading to substantial memory overhead. In this work, we propose a new algorithm for correlated-noise private training that correlates noise only with the immediately preceding iteration and cancels a tunable portion of it. By regenerating the previous noise using a pseudorandom number generator rather than storing it, our method has essentially the same memory footprint as DP-SGD. We show that the computational overhead is minimal and empirically demonstrate improved accuracy over DP-SGD.
Machine Learning (cs.LG)
arXiv:2601.22334 [cs.LG]
(or arXiv:2601.22334v3 [cs.LG] for this version)
From: Nikita P. Kalinin [view email] [v1] Thu, 29 Jan 2026 21:21:34 UTC (230 KB) [v2] Mon, 11 May 2026 20:34:37 UTC (230 KB) [v3] Sat, 26 Sep 2026 20:16:25 UTC (259 KB)
Full text under the publisher's open licence · 165 words
Related coverage
Beyond Square Roots: A Memory-Efficient Explicit Factorization for Multi-Epoch Private Learning
arXiv · Sep 30, 2026
Cocoon: A System Architecture for Differentially Private Training with Correlated Noises
arXiv · Oct 6, 2026
News
Quantum Computing Report · Jun 7, 2026
Netskope (NTSK) Q2 2027 Earnings Call Transcript
fool.com · Sep 10, 2026