Skip to content
aiqbench takeAI Safety

aiqbench analysis

This research introduces a method to enhance the reliability of language models in constraint solving by replacing generic error messages with minimal unsatisfiable cores derived from Answer Set Programming. By providing precise proofs rather than just failure signals, this approach drastically reduces solution fabrication rates and offers verifiable certificates for generated outputs. The work highlights that while strong chain-of-thought models can match symbolic accuracy on their own, the primary value of integrating formal solvers lies in generating rigorous guarantees against hallucination.
Source coveringarXiv
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

From Errors to Proofs: Minimal-Core-Guided Repair for Neuro-Symbolic Constraint Solving

arXiv2 min read

arXiv:2608.14771v1 Announce Type: cross Abstract: Making language models solve constraint problems reliably often means having them translate the problem into a formal specification and delegating the search to a sound solver. But the translation is itself a language-model task, and an unfaithful translation makes the solver faithfully solve the wrong problem. Existing pipelines repair only translations that crash, returning the solver's error message and falling silent when the program runs but is wrong. We replace the error message with a proof: when the generated program is unsatisfiable, we extract a minimal unsatisfiable core over the model's own constraints and hand it back the exact set that cannot hold together, a leakage-free signal that localizes the fault. On a new benchmark of 77 problems with an exact oracle, translation to Answer Set Programming is faithful on six of seven domains and fails only on aggregate coverage scheduling, which concentrates the translation tax in one diagnosable pattern. A minimal core, rather than a bare error, is what stops a weaker model from fabricating solutions to infeasible problems, cutting fabrication from 79% to 7%. A strong chain-of-thought baseline meanwhile matches the symbolic route on accuracy, so the route's value is not accuracy but certificates and its refusal to fabricate.

Abstract:Making language models solve constraint problems reliably often means having them translate the problem into a formal specification and delegating the search to a sound solver. But the translation is itself a language-model task, and an unfaithful translation makes the solver faithfully solve the wrong problem. Existing pipelines repair only translations that crash, returning the solver's error message and falling silent when the program runs but is wrong. We replace the error message with a proof: when the generated program is unsatisfiable, we extract a minimal unsatisfiable core over the model's own constraints and hand it back the exact set that cannot hold together, a leakage-free signal that localizes the fault. On a new benchmark of 77 problems with an exact oracle, translation to Answer Set Programming is faithful on six of seven domains and fails only on aggregate coverage scheduling, which concentrates the translation tax in one diagnosable pattern. A minimal core, rather than a bare error, is what stops a weaker model from fabricating solutions to infeasible problems, cutting fabrication from 79% to 7%. A strong chain-of-thought baseline meanwhile matches the symbolic route on accuracy, so the route's value is not accuracy but certificates and its refusal to fabricate.

7 pages, 2 figures. Accepted at the IJCAI-ECAI 2026 Workshop on Logic and Symbolic Reasoning (LogiSymb), poster

Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Logic in Computer Science (cs.LO); Programming Languages (cs.PL); Symbolic Computation (cs.SC); Optimization and Control (math.OC)

arXiv:2608.14771 [cs.AI]

(or arXiv:2608.14771v1 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2608.14771 arXiv-issued DOI via DataCite (pending registration)

Full text under the publisher's open licence · 258 words

Related coverage