analiza aiqbench
This arXiv preprint proposes CausalCoT, an inference method that limits each reasoning step's attention to its causal parents, aiming to make chain-of-thought explanations reflect the reasoning that actually drives the answer. The reported gains in causal sensitivity are measured against baselines the authors chose, so readers should check the benchmarks and evaluation protocol before drawing conclusions. For anyone working on explanation faithfulness, the approach is a concrete, testable attempt to tie generated rationales to the model's actual computation.
Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation
arXiv:2610.31639v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in which local corrections propagate predictably through downstream reasoning.
Abstract:Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In standard autoregressive generation, dense attention allows each reasoning step to attend to the full context, enabling the model to derive answers through direct lookup while generating plausible-sounding but post-hoc justifications. We introduce CausalCoT, an inference framework that enforces causal transparency by decomposing problems into a directed acyclic graph of atomic sub-questions, evaluating each node in an isolated context containing only its direct causal parents. This Markovian structure removes attention pathways to non-parent content, making downstream predictions sensitive to local reasoning rather than global context leakage. Across a diverse suite of reasoning benchmarks, CausalCoT preserves competitive predictive performance while improving explanation faithfulness, increasing causal sensitivity of intermediate reasoning steps by an average of 12 and up to 21 percentage points over the strongest baseline. The structured dependency graph additionally enables test-time interventions in which local corrections propagate predictably through downstream reasoning.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2610.31639 [cs.LG]
(or arXiv:2610.31639v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.31639 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 262 cuvinte
Subiecte conexe
Hybrid Gated Attention
arXiv · 15 oct. 2026
Mitigating Object Hallucinations in LVLMs via Attention Imbalance Rectification
arXiv · 16 iun. 2026
Multilinguality in Hybrid Attention LLMs
arXiv · 30 sept. 2026
More Value per Key: Asymmetric Sparse Attention for Faster LLM Decoding
arXiv · 6 oct. 2026