aiqbench analysis
The article focuses on PUPPET, a novel framework using Direct Preference Optimization (DPO) to simultaneously enhance the detectability of LLM outputs and maintain high performance on downstream tasks like summarization and QA. This approach resolves the traditional trade-off where watermarking degrades model utility by integrating detection signals directly into the generation process without sacrificing quality. The method demonstrates robustness across various detector architectures, model sizes, and even against paraphrasing attacks, offering a practical solution for ensuring transparency in deployed AI systems.
LLM Output Detectability and Task Performance Can be Jointly Optimized
arXiv:2605.01350v2 Announce Type: replace Abstract: Detecting machine-generated text is essential for transparency and accountability when deploying LLMs. Watermarking enables statistically reliable detection by biasing token distributions to embed detectable signals into LLM outputs. However, it has been reported that watermarked LLMs often perform worse on downstream tasks. We propose PUPPET, a framework that fine-tunes an LLM via DPO to generate text that is both more detectable by a target detector and better performing on downstream tasks. We use two rewards: a detector that outputs a machine-class likelihood and an evaluator that measures a task-specific metric. Just as a watermark is verified with its secret key, this detector-specific design lets an LLM provider track how its published model is used. Experiments on long-form QA, summarization, and essay writing show that LLMs trained with PUPPET achieve detectability competitive with watermarking methods---even at strict low FPRs---while outperforming them on downstream tasks. Moreover, this optimization requires only a few thousand samples and 1--2 GPU hours, and its gains hold across out-of-domain tasks, six detectors of diverse architectures, and different LLM families and sizes, and are even robust to paraphrasing attacks.
Abstract:Detecting machine-generated text is essential for transparency and accountability when deploying LLMs. Watermarking enables statistically reliable detection by biasing token distributions to embed detectable signals into LLM outputs. However, it has been reported that watermarked LLMs often perform worse on downstream tasks. We propose PUPPET, a framework that fine-tunes an LLM via DPO to generate text that is both more detectable by a target detector and better performing on downstream tasks. We use two rewards: a detector that outputs a machine-class likelihood and an evaluator that measures a task-specific metric. Just as a watermark is verified with its secret key, this detector-specific design lets an LLM provider track how its published model is used. Experiments on long-form QA, summarization, and essay writing show that LLMs trained with PUPPET achieve detectability competitive with watermarking methods---even at strict low FPRs---while outperforming them on downstream tasks. Moreover, this optimization requires only a few thousand samples and 1--2 GPU hours, and its gains hold across out-of-domain tasks, six detectors of diverse architectures, and different LLM families and sizes, and are even robust to paraphrasing attacks.
15 pages, 9 figures, 20 tables. Code: this https URL
Computation and Language (cs.CL)
arXiv:2605.01350 [cs.CL]
(or arXiv:2605.01350v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.01350 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 284 words
Related coverage
Links #5: 2026/07
LessWrong · Jul 31, 2026
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
Introducing CARE-X: Towards Clinically Useful Radiology VLMs with Auxiliary Supervision, Reward-Aligned Learning, and Tool-Augmented Measurement
Microsoft Research · Aug 11, 2026
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · Sep 4, 2026