analiza aiqbench
This research addresses the critical tension between differential privacy and model merging, identifying specific geometric obstacles like local sharpness and reference drift that hinder combining private task models. The proposed DP-Merging framework mitigates these issues by guiding models toward flatter loss regions and aligning them with a shared initialization, thereby improving performance without compromising privacy guarantees. This work is essential for advancing secure, collaborative AI development where data centralization is impossible.
When Privacy Hurts Mergeability: Geometry-Aware Model Merging under Differential Privacy
arXiv:2608.26655v1 Announce Type: new Abstract: Model merging promises to construct a single multi-task model from independently fine-tuned task models without accessing the original task data. This makes it attractive when task data cannot be centralized, but released task models may still leak private fine-tuning data. Differential privacy (DP) provides a principled mechanism for limiting such leakage, yet its effect on model merging remains poorly understood. In this paper, we study the geometry of differentially private model merging and identify two geometric obstacles that make private task models difficult to merge: \emph{local sharpness}, which makes task losses sensitive to the parameter displacement induced by merging, and \emph{reference drift}, which measures the displacement of private task models from the shared pretrained initialization and amplifies cross-task interference. Based on these observations, we propose \textbf{DP-Merging}, a geometry-aware framework that improves the mergeability of differentially private task models. DP-Merging uses a DP-compatible sharpness-aware objective to guide each private task model toward flatter loss regions, and a reference-based alignment regularizer to keep task models close to the shared pretrained initialization. We derive a merge-gap upper bound showing that reducing local curvature and reference drift tightens the bound on the loss increase induced by merging. Experiments on vision and language tasks across multiple privacy budgets show that DP-Merging consistently improves private merged-model performance while preserving the privacy guarantees of the underlying DP fine-tuning procedures.
Abstract:Model merging promises to construct a single multi-task model from independently fine-tuned task models without accessing the original task data. This makes it attractive when task data cannot be centralized, but released task models may still leak private fine-tuning data. Differential privacy (DP) provides a principled mechanism for limiting such leakage, yet its effect on model merging remains poorly understood. In this paper, we study the geometry of differentially private model merging and identify two geometric obstacles that make private task models difficult to merge: \emph{local sharpness}, which makes task losses sensitive to the parameter displacement induced by merging, and \emph{reference drift}, which measures the displacement of private task models from the shared pretrained initialization and amplifies cross-task interference. Based on these observations, we propose \textbf{DP-Merging}, a geometry-aware framework that improves the mergeability of differentially private task models. DP-Merging uses a DP-compatible sharpness-aware objective to guide each private task model toward flatter loss regions, and a reference-based alignment regularizer to keep task models close to the shared pretrained initialization. We derive a merge-gap upper bound showing that reducing local curvature and reference drift tightens the bound on the loss increase induced by merging. Experiments on vision and language tasks across multiple privacy budgets show that DP-Merging consistently improves private merged-model performance while preserving the privacy guarantees of the underlying DP fine-tuning procedures.
Machine Learning (cs.LG)
arXiv:2608.26655 [cs.LG]
(or arXiv:2608.26655v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.26655 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 316 cuvinte
Subiecte conexe
AlignMerge - Alignment-Preserving Large Language Model Merging via Fisher-Guided Geometric Constraints
arXiv · 15 oct. 2026
PermDoRA -- Understanding Adapter Interference in Language Models: Limits of Parameter-Space Geometry
arXiv · 12 iun. 2026
CAMFT: Conflict-Aware Mergeable Fine-Tuning for Large Language Models
arXiv · 22 sept. 2026
SLEM: Spectrum-Aware Latent Expert Merging
arXiv · 15 oct. 2026