OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets

Abstract:When a data contributor requests removal, model trainers face a practical gap: unlearning algorithms require a forget set, yet no tool can locate which training records belong to a given author. Existing provenance systems operate at file or dataset level, forcing catastrophic over-deletion. We present ob, a record- and token-level data provenance system that propagates author identity through data processing pipelines and resolves revocation requests into precise forget sets via deterministic queries. Evaluation on 219,555 Wikipedia pages demonstrates that record-level provenance eliminates dataset-level over-deletion (from 101x to 1.3x), while integration adds 1.3-4.0% throughput overhead (HuggingFace) and 2.1-19.0% (Datatrove) on wiki data. On a 1.7B model, provenance-based forget sets consistently reduce the collateral damage of machine unlearning (retain perplexity) relative to same-size random baselines across all evaluated authors, with membership-inference tests indicating they select genuinely memorized content.

Artificial Intelligence (cs.AI)

arXiv:2607.13037 [cs.AI]

(or arXiv:2607.13037v2 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2607.13037 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.