aiqbench analysis
This research exposes a critical vulnerability in AI debate systems where a single round of discussion can artificially manufacture consensus without improving accuracy, effectively silencing dissenting signals needed for error detection. The study demonstrates that verification mechanisms relying on post-debate unanimity are dangerously flawed, as the collapse of disagreement occurs independently of actual correctness improvements or peer persuasion. For developers building reliable LLM evaluation pipelines, this finding mandates verifying model outputs before any debate exposure to prevent the erosion of essential safety checks.
Unanimity Without Persuasion: A Single Round of Debate Erases the Disagreement That Verification Needs
arXiv:2609.26145v1 Announce Type: new Abstract: A debate panel can become unanimous without becoming more correct. This is dangerous for downstream safeguards: a substituted verification ballot can change only narrow-margin votes, while richer arbiters lose disagreement as a natural targeting signal. We show that one debate round can erase that resource without requiring persuasion. Tracking a heterogeneous 7-judge panel through a blind round and three debate rounds on 600 code-correctness candidates, unanimity on a fixed cohort jumps from 39.5\% to 95.2\% in round 1 (93.1\% of the total collapse), while accuracy moves by less than one point and 96.3\% of verdict flips follow the displayed peer majority. An execution-based verification ballot corrects 8 of 2,037 pre-debate candidate-substitution instances but changes zero in every later round; by round 3 every wrong decision is unanimous, erasing dissent that had flagged two-thirds of the panel's errors. Identical-cohort controls explain why: no-peer reconsideration reproduces 79.6\% of the collapse, real labels without reasoning reproduce 91.5\%, and random labels steer flips toward whatever they display; the full-debate condition adds 4.3 percentage points over labels only (clustered 95\% CI 0.7--8.1). The one-round collapse reproduces in two additional real runs and two fake-label seeds, remains under panel sizes 3--7, and appears in MATH-500. Parse failures concentrate on contested candidates ($p<0.001$), making attrition non-ignorable. The design implication is operational: verify before any second-pass evaluation or peer exposure, and never treat post-debate unanimity as independent evidence of reliability.
Abstract:A debate panel can become unanimous without becoming more correct. This is dangerous for downstream safeguards: a substituted verification ballot can change only narrow-margin votes, while richer arbiters lose disagreement as a natural targeting signal. We show that one debate round can erase that resource without requiring persuasion. Tracking a heterogeneous 7-judge panel through a blind round and three debate rounds on 600 code-correctness candidates, unanimity on a fixed cohort jumps from 39.5\% to 95.2\% in round 1 (93.1\% of the total collapse), while accuracy moves by less than one point and 96.3\% of verdict flips follow the displayed peer majority. An execution-based verification ballot corrects 8 of 2,037 pre-debate candidate-substitution instances but changes zero in every later round; by round 3 every wrong decision is unanimous, erasing dissent that had flagged two-thirds of the panel's errors. Identical-cohort controls explain why: no-peer reconsideration reproduces 79.6\% of the collapse, real labels without reasoning reproduce 91.5\%, and random labels steer flips toward whatever they display; the full-debate condition adds 4.3 percentage points over labels only (clustered 95\% CI 0.7--8.1). The one-round collapse reproduces in two additional real runs and two fake-label seeds, remains under panel sizes 3--7, and appears in MATH-500. Parse failures concentrate on contested candidates ($p<0.001$), making attrition non-ignorable. The design implication is operational: verify before any second-pass evaluation or peer exposure, and never treat post-debate unanimity as independent evidence of reliability.
Artificial Intelligence (cs.AI)
arXiv:2609.26145 [cs.AI]
(or arXiv:2609.26145v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.26145 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 326 words
Related coverage
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026
The current bottleneck is political will, not research
AI Alignment Forum · Jul 11, 2026
The Continual Constitutional Convention: How language models revise the documents that govern them
AI Alignment Forum · Oct 13, 2026

Lattice-Based Cryptography Explained
Quantum Zeitgeist · Aug 15, 2026