analiza aiqbench
The paper treats AI text detection as a distribution-shift problem: supervised detectors degrade under adversarial humanization, new LLM releases, and drift in human writing. Its proposed remedy is test-time adaptation that uses homogeneity among unlabeled inference-time samples. The reported gap (27.0% vs. 93.3% detection of adversarial AI text for the commercial Pangram detector versus the authors' method) is striking, but it comes from the authors' own benchmark, so it needs independent replication before readers treat it as a general result.
Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift
arXiv:2606.25152v2 Announce Type: replace-cross Abstract: Deployed AI text detection approaches are supervised, relying on training-time access to labeled datasets of both human-written and AI-generated text. This approach is vulnerable to three types of distribution shifts that occur continually post-deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and unidentifiable temporal drift in human writing after widespread LLM usage. Simultaneously, existing approaches do not leverage a key signal of LLM usage: inference-time homogeneity. We propose a test-time adaptation (TTA) approach that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time. Empirically, we find that state-of-the-art supervised detectors systematically fail when they encounter distribution shifts in AI-generated and human writing, both adversarial and natural, while TTA with semi-supervised learning is largely robust; e.g., the commercial model Pangram detects just 27.0% of our adversarial AI-generated text, compared to 93.3% for our test-time approach. We establish that TTA is a promising framework for AI text detection in the wild.
Abstract:Deployed AI text detection approaches are supervised, relying on training-time access to labeled datasets of both human-written and AI-generated text. This approach is vulnerable to three types of distribution shifts that occur continually post-deployment, and for which labeled data is often unavailable: adversarial humanization, new LLMs being released, and unidentifiable temporal drift in human writing after widespread LLM usage. Simultaneously, existing approaches do not leverage a key signal of LLM usage: inference-time homogeneity. We propose a test-time adaptation (TTA) approach that adapts to distribution shifts by leveraging homogeneity among unlabeled samples observed at inference time. Empirically, we find that state-of-the-art supervised detectors systematically fail when they encounter distribution shifts in AI-generated and human writing, both adversarial and natural, while TTA with semi-supervised learning is largely robust; e.g., the commercial model Pangram detects just 27.0% of our adversarial AI-generated text, compared to 93.3% for our test-time approach. We establish that TTA is a promising framework for AI text detection in the wild.
NeurIPS 2026. We release our code at: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.25152 [cs.CL]
(or arXiv:2606.25152v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.25152 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 267 cuvinte
Subiecte conexe
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
An unexamined cause of the OpenAI Hugging Face hacking incident: its binary performance metric
LessWrong · 23 sept. 2026
2026.9: There's room on this bus
Home Assistant · 2 sept. 2026
Problems of Proliferation
LessWrong · 29 sept. 2026