aiqbench analysis
This research demonstrates how deep Transformers can achieve explicit Chain-of-Thought performance through implicit reasoning over Horn clauses, provided specific architectural constraints like bidirectional prefix masks are met. While this approach effectively handles various graph structures and problem widths without relying on spurious correlations, the study notes that standard CoT remains essential for extrapolating to greater model depths.
The Scaling Properties of Implicit Deductive Reasoning in Transformers
arXiv:2605.04330v2 Announce Type: replace Abstract: We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By systematically decorrelating provability from spurious features and enforcing algorithmic alignment, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation.
Abstract:We investigate the scaling properties of implicit deductive reasoning over Horn clauses in depth-bounded Transformers. By systematically decorrelating provability from spurious features and enforcing algorithmic alignment, we find that in sufficiently deep models with a bidirectional prefix mask, implicit reasoning approaches explicit CoT performance across graph topologies and problem widths, though CoT remains necessary for depth extrapolation.
Artificial Intelligence (cs.AI); Computational Complexity (cs.CC); Logic in Computer Science (cs.LO); Symbolic Computation (cs.SC)
arXiv:2605.04330 [cs.AI]
(or arXiv:2605.04330v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2605.04330 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 160 words
