aiqbench analysis
This research exposes a critical vulnerability in current machine-generated text detectors, revealing that their high accuracy on scientific benchmarks often stems from superficial PDF formatting artifacts rather than genuine linguistic analysis. By introducing format counterfactuals and normalizing document structures before training, the study demonstrates a significant reduction in generalization gaps and improved performance on out-of-domain data. These findings are essential for developing more robust detection systems that rely on semantic content instead of extraction-side effects.
Measuring Artifact Reliance in Machine-Generated Text Detectors
arXiv:2610.13583v1 Announce Type: new Abstract: Detectors of machine-generated text score almost perfectly on scientific-literature benchmarks, implying we can reliably distinguish with these detectors human and machine-generated documents. We looked at ten scientific-document subsets from six popular resources, and on several of them that apparent ease is a side effect of how the documents were put together. A logistic regression using 27 superficial features reached 0.833-0.997 macro-F1, and on the three arXiv subsets it outperformed every detector we evaluate with F1-scores 0.951-0.997. The strongest cue to the detector is the PDF extraction artifact: human texts retain their original line breaks, whereas generated texts are extracted as a single block of prose. This cue, together with three other formatting features, tell the classes apart at 0.996 AUC. To distinguish benchmarks that contain such a signal from detectors that rely on one, we introduce format counterfactuals: just re-wrapping a machine-written abstract without changing a single word reverses 20-27% of a public detector's correct decisions. Normalizing formatting before training reduces the generalization gap from 0.32 to 0.22. On format-clean benchmarks, it also raises out-of-domain macro-F1 from 0.33 to 0.65 for detectors trained on artifact-containing subsets.
Abstract:Detectors of machine-generated text score almost perfectly on scientific-literature benchmarks, implying we can reliably distinguish with these detectors human and machine-generated documents. We looked at ten scientific-document subsets from six popular resources, and on several of them that apparent ease is a side effect of how the documents were put together. A logistic regression using 27 superficial features reached 0.833-0.997 macro-F1, and on the three arXiv subsets it outperformed every detector we evaluate with F1-scores 0.951-0.997. The strongest cue to the detector is the PDF extraction artifact: human texts retain their original line breaks, whereas generated texts are extracted as a single block of prose. This cue, together with three other formatting features, tell the classes apart at 0.996 AUC. To distinguish benchmarks that contain such a signal from detectors that rely on one, we introduce format counterfactuals: just re-wrapping a machine-written abstract without changing a single word reverses 20-27% of a public detector's correct decisions. Normalizing formatting before training reduces the generalization gap from 0.32 to 0.22. On format-clean benchmarks, it also raises out-of-domain macro-F1 from 0.33 to 0.65 for detectors trained on artifact-containing subsets.
Computation and Language (cs.CL)
arXiv:2610.13583 [cs.CL]
(or arXiv:2610.13583v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.13583 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 281 words
Related coverage
VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMs
arXiv · Sep 1, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026
Coverage-driven alignment - What ‘Teaching Claude Why’ can borrow from AV verification
LessWrong · Jun 8, 2026
Hyperloom: A Multi-Agent Harness for Autonomous Inference Optimization on AMD GPUs
AMD · Sep 21, 2026