aiqbench analysis
This research introduces Quadratic Objective Perturbation (QOP), a novel differentially private mechanism that leverages curvature to relax strict gradient boundedness assumptions common in modern machine learning. By controlling sensitivity through spectral properties rather than linear terms, the method enhances stability and utility for models where traditional approaches fail. Theoretical analysis confirms preserved privacy guarantees even under approximate optimization solutions, offering a more robust framework for empirical risk minimization.
Quadratic Objective Perturbation: Curvature-Based Differential Privacy
arXiv:2605.05905v2 Announce Type: replace Abstract: Objective perturbation is a standard mechanism in differentially private empirical risk minimization. In particular, Linear Objective Perturbation (LOP) enforces privacy by adding a random linear term, while strong convexity and stability are ensured by an additional deterministic quadratic term. However, this approach requires the strong assumption of bounded gradients of the loss function, which excludes many modern machine learning models. In this work, we introduce Quadratic Objective Perturbation (QOP), which perturbs the objective with a random quadratic form. This perturbation induces strong convexity and enforces stability of the problem through curvature, thereby enabling privacy and allowing sensitivity to be controlled through spectral properties of the perturbation rather than assumptions on the gradients. As a result, we obtain $(\varepsilon, \delta)$-differential privacy under weaker \red{gradient} assumptions. Furthermore, we extend the analysis to account for approximate solutions, showing that privacy guarantees are preserved under inexact solves. Additionally, we derive utility guarantees in terms of empirical excess risk, and provide a theoretical and numerical comparison to LOP, highlighting the advantages of curvature-based perturbations. Finally, we discuss algorithmic aspects and show that the resulting problems can be solved efficiently using modern splitting schemes.
Abstract:Objective perturbation is a standard mechanism in differentially private empirical risk minimization. In particular, Linear Objective Perturbation (LOP) enforces privacy by adding a random linear term, while strong convexity and stability are ensured by an additional deterministic quadratic term. However, this approach requires the strong assumption of bounded gradients of the loss function, which excludes many modern machine learning models. In this work, we introduce Quadratic Objective Perturbation (QOP), which perturbs the objective with a random quadratic form. This perturbation induces strong convexity and enforces stability of the problem through curvature, thereby enabling privacy and allowing sensitivity to be controlled through spectral properties of the perturbation rather than assumptions on the gradients. As a result, we obtain $(\varepsilon, \delta)$-differential privacy under weaker \red{gradient} assumptions. Furthermore, we extend the analysis to account for approximate solutions, showing that privacy guarantees are preserved under inexact solves. Additionally, we derive utility guarantees in terms of empirical excess risk, and provide a theoretical and numerical comparison to LOP, highlighting the advantages of curvature-based perturbations. Finally, we discuss algorithmic aspects and show that the resulting problems can be solved efficiently using modern splitting schemes.
Machine Learning (cs.LG); Optimization and Control (math.OC)
arXiv:2605.05905 [cs.LG]
(or arXiv:2605.05905v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2605.05905 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 284 words
Related coverage
Explaining f-Divergence-Based Regularization via Local Curvature and Sharpness-Aware Minimization
arXiv · Sep 10, 2026
Stabilizing Private LASSO under Heterogeneous Covariates via Anisotropic Objective Perturbation
arXiv · Sep 3, 2026
When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy
arXiv · Aug 28, 2026
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026