analiza aiqbench
This paper tackles hallucinated structural and property claims in LLM chemical reasoning by pairing generation with a Lean-based kernel that checks each molecular state transition. Reported gains come from whole-trace refinement on 8,800 examples with one model (Qwen3.6-27B), and the MCP interface helps mainly on reaction tasks. For AI readers, the useful idea is verifiable, revisable intermediate commitments, though the evidence is limited to one model family and needs replication.
MolLean: A Process-Verifiable Language for Chemical Reasoning
arXiv:2610.20459v1 Announce Type: new Abstract: Chemical chain-of-thought can contain hallucinated structural and property claims. We thereby introduce MolLean, a process-verifiable language that grounds chemical reasoning in an explicit molecular state. A layered Lean-based kernel checks state transitions and assertions over molecular readouts, from primitive graph edits to compound transformations. Meanwhile, a typed witness protocol separates native structural checks from externally supplied measurements. We further introduce MolLean-MCP, a stateful tool interface that exposes these transitions, checks, and readouts during generation: the model receives feedback to decide its next action, and rejected actions leave the state unchanged. MolLean supports one-pass trace execution, whole-trace refinement, and interactive reasoning through MolLean-MCP. We evaluate these three modes with direct chain-of-thought on 8,800 matched examples across 12 tasks spanning molecular generation, editing, optimization, and reactions. With Qwen3.6-27B, whole-trace refinement outperforms direct chain-of-thought on all 12 tasks. MolLean-MCP complements whole-trace refinement with stepwise verification feedback, reaching better performance in reaction tasks. Together, these results highlight an executable foundation for chemical reasoning, in which explicit chemical commitments can be checked and revised rather than remaining solely in free-form explanations.
Abstract:Chemical chain-of-thought can contain hallucinated structural and property claims. We thereby introduce MolLean, a process-verifiable language that grounds chemical reasoning in an explicit molecular state. A layered Lean-based kernel checks state transitions and assertions over molecular readouts, from primitive graph edits to compound transformations. Meanwhile, a typed witness protocol separates native structural checks from externally supplied measurements. We further introduce MolLean-MCP, a stateful tool interface that exposes these transitions, checks, and readouts during generation: the model receives feedback to decide its next action, and rejected actions leave the state unchanged. MolLean supports one-pass trace execution, whole-trace refinement, and interactive reasoning through MolLean-MCP. We evaluate these three modes with direct chain-of-thought on 8,800 matched examples across 12 tasks spanning molecular generation, editing, optimization, and reactions. With Qwen3.6-27B, whole-trace refinement outperforms direct chain-of-thought on all 12 tasks. MolLean-MCP complements whole-trace refinement with stepwise verification feedback, reaching better performance in reaction tasks. Together, these results highlight an executable foundation for chemical reasoning, in which explicit chemical commitments can be checked and revised rather than remaining solely in free-form explanations.
Computation and Language (cs.CL); Computational Engineering, Finance, and Science (cs.CE); Programming Languages (cs.PL)
arXiv:2610.20459 [cs.CL]
(or arXiv:2610.20459v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.20459 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 280 cuvinte
Subiecte conexe
From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models
arXiv · 4 iun. 2026
From Answers to States: Verifiable Process-Level Evaluation of Chemical Reasoning in Large Language Models
arXiv · 3 iun. 2026
ChemVTS-Bench: Evaluating Visual-Textual-Symbolic Reasoning of Multimodal Large Language Models in Chemistry
arXiv · 22 sept. 2026
Podcast with Johannes Pollanen, Co-founder of EeroQ and a Professor of Physics at Michigan State University
Quantum Computing Report · 17 oct. 2026