analiza aiqbench
The paper targets a real gap in AI for mathematics: most work trains LLMs to prove true statements, while this one trains them to find counterexamples to false ones, with outputs checked automatically in Lean 4. Its main technical contributions are a symbolic mutation method that generates training tasks by removing hypotheses from theorems, and a multi-reward expert iteration framework. The evidence is limited to three newly collected benchmarks from the authors, so readers should look for independent replication and for how well the gains transfer beyond the authors' own datasets.
Learning to Disprove: Formal Counterexample Generation with Large Language Models
arXiv:2603.19514v2 Announce Type: replace Abstract: Mathematical reasoning demands two critical, complementary skills: constructing rigorous proofs for true statements and discovering counterexamples that disprove false ones. However, current AI efforts in mathematics focus almost exclusively on proof construction, often neglecting the equally important task of finding counterexamples. In this paper, we address this gap by fine-tuning large language models (LLMs) to reason about and generate counterexamples. We formalize this task as formal counterexample generation, which requires LLMs not only to propose candidate counterexamples but also to produce formal proofs that can be automatically verified in the Lean 4 theorem prover. To enable effective learning, we introduce a symbolic mutation strategy that constructs candidate counterexample-generation tasks by systematically extracting theorems and removing selected hypotheses. Together with curated datasets, this strategy enables a multi-reward expert iteration framework that substantially enhances both the effectiveness and efficiency of training LLMs for counterexample generation and theorem proving. Experiments on three newly collected benchmarks validate the advantages of our approach, showing that the mutation strategy and training framework yield substantial performance gains.
Abstract:Mathematical reasoning demands two critical, complementary skills: constructing rigorous proofs for true statements and discovering counterexamples that disprove false ones. However, current AI efforts in mathematics focus almost exclusively on proof construction, often neglecting the equally important task of finding counterexamples. In this paper, we address this gap by fine-tuning large language models (LLMs) to reason about and generate counterexamples. We formalize this task as formal counterexample generation, which requires LLMs not only to propose candidate counterexamples but also to produce formal proofs that can be automatically verified in the Lean 4 theorem prover. To enable effective learning, we introduce a symbolic mutation strategy that constructs candidate counterexample-generation tasks by systematically extracting theorems and removing selected hypotheses. Together with curated datasets, this strategy enables a multi-reward expert iteration framework that substantially enhances both the effectiveness and efficiency of training LLMs for counterexample generation and theorem proving. Experiments on three newly collected benchmarks validate the advantages of our approach, showing that the mutation strategy and training framework yield substantial performance gains.
Artificial Intelligence (cs.AI)
arXiv:2603.19514 [cs.AI]
(or arXiv:2603.19514v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2603.19514 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 262 cuvinte
Subiecte conexe
[AINews] not much happened today
Latent Space · 10 sept. 2026
[AINews] OpenAI reports Navier-Stokes singularity find in 88 hours using Astra-next, roughly 10,000 agents and 130B tokens (>$40M), a contender for second ever Millennium Prize awarded
Latent Space · 9 sept. 2026
SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?
arXiv · 22 sept. 2026
Open-Source LLM-Driven Formal Verification: A Multi-Agent Pipeline for RTL Repair
arXiv · 3 aug. 2026