OVIG: Optimistic Verification of AI Training Integrity via Gradient Signals

Abstract:The rapid growth of AI has increased the demand for domain-specific models. Post-training of open-source models offers a more economical way to meet this growing demand, but the cost of accelerator infrastructure often pushes organizations to outsource the process to third-party providers. An untrusted provider may deviate from the declared training procedure to save computation or inject malicious behavior. A potential solution is to audit the training by having an independent verifier replay the training and compare the results. This faces two key challenges: benign numerical drift from floating-point computation across heterogeneous accelerators is difficult to distinguish from malicious deviations, and the checkpoints and metadata required for verification introduce substantial storage and transmission costs. Existing approaches either have security limitations or high deployment costs. We present OVIG, an optimistic verification framework that verifies training using an empirical boundary on gradient differences calibrated from honest heterogeneous replay. Gradient differences that exceed this boundary are treated as malicious deviations. OVIG further uses optimistic verification and partitions training into stride-$s$ intervals, retaining evidence only at interval endpoints to reduce storage and transmission costs. Across shortcut training attacks and adaptive targeted manipulation attacks, OVIG maintains \(0\%\) ASR on language, vision, and diffusion workloads. On Qwen3, increasing the stride from \(s=1\) to \(s=2000\) reduces off-chain storage and evidence transmission by \(1996\times\) while preserving \(0\%\) ASR and only adds \(14.3\%\) system computation overhead relative to unverified training.

Cryptography and Security (cs.CR); Machine Learning (cs.LG)

arXiv:2606.21045 [cs.CR]

(or arXiv:2606.21045v2 [cs.CR] for this version)

https://doi.org/10.48550/arXiv.2606.21045 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.