analiza aiqbench
The study tests whether membership-inference attacks can reliably detect whether a language model was trained on a specific document. By re-running OLMo 2 1B's mid-training with randomized, matched inserts and using the released run as a counterfactual, it shows that a single exposure barely shifts individual document scores (3.8% of members caught at 1% FPR, despite 0.767 AUC). Collection-level detection is far more reliable, which matters for anyone assessing copyright, privacy, or contamination claims about LLMs.
From Training Exposure to Membership Evidence: A Controlled Study of OLMo 2
arXiv:2610.41732v1 Announce Type: cross Abstract: Whether a language model was trained on a document matters for copyright, privacy, and benchmark contamination. Yet most membership-inference (MI) evaluations on language models collect members and non-members after the fact, so success may reflect distribution shift and failure is inconclusive. We re-ran OLMo 2 1B's official 50-billion-token mid-training stage, replacing selected training chunks with documents from 7,814 pairs matched on domain and difficulty. For each pair, a coin flip determined which document was inserted and which was held out. The run released by the Allen Institute for AI (AI2) from the same seed saw identical data everywhere else and serves as the counterfactual. One exposure raises a previously unseen document's likelihood on average but rarely identifies it. Using the counterfactual as reference, the reference ratio has an area under the ROC curve (AUC) of 0.767 but, with one threshold across domains, catches 3.8% of members at a 1% false-positive rate (FPR). In contrast, 13 matched pairs give a paired test 80% power at the 5% level to detect training on a collection, given same-domain non-members and a close reference. A reference trained with another seed recovers 92% of the counterfactual's AUC advantage over chance, whereas reference-free attacks stay within 0.07 of chance when pooled over domains. The full official post-training recipe, re-run on both models, leaves 89% of the average effect. Even with a closely matched reference, substantial document-level variability limits individual membership inference. We will release the checkpoints, per-token log-probabilities, and the evaluation documents; the code is available at https://github.com/ali-bana/OLMo-Twin.
Abstract:Whether a language model was trained on a document matters for copyright, privacy, and benchmark contamination. Yet most membership-inference (MI) evaluations on language models collect members and non-members after the fact, so success may reflect distribution shift and failure is inconclusive. We re-ran OLMo 2 1B's official 50-billion-token mid-training stage, replacing selected training chunks with documents from 7,814 pairs matched on domain and difficulty. For each pair, a coin flip determined which document was inserted and which was held out. The run released by the Allen Institute for AI (AI2) from the same seed saw identical data everywhere else and serves as the counterfactual. One exposure raises a previously unseen document's likelihood on average but rarely identifies it. Using the counterfactual as reference, the reference ratio has an area under the ROC curve (AUC) of 0.767 but, with one threshold across domains, catches 3.8% of members at a 1% false-positive rate (FPR). In contrast, 13 matched pairs give a paired test 80% power at the 5% level to detect training on a collection, given same-domain non-members and a close reference. A reference trained with another seed recovers 92% of the counterfactual's AUC advantage over chance, whereas reference-free attacks stay within 0.07 of chance when pooled over domains. The full official post-training recipe, re-run on both models, leaves 89% of the average effect. Even with a closely matched reference, substantial document-level variability limits individual membership inference. We will release the checkpoints, per-token log-probabilities, and the evaluation documents; the code is available at this https URL.
20 pages, 11 figures. Code: this https URL
Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2610.41732 [cs.CR]
(or arXiv:2610.41732v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2610.41732 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 364 cuvinte
Subiecte conexe
Contain the Chaos: ‘CONTROL Resonant’ Launches on GeForce NOW
NVIDIA · 24 sept. 2026
COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration
arXiv · 14 oct. 2026
IsoAct: Geometry-Aware Post-hoc Debiasing via Isometric Actions
arXiv · 27 oct. 2026
GPU-Accelerated Clustering for Financial Instruments at Scale
NVIDIA · 21 aug. 2026