What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation

Title:What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activation

Abstract:Chain-of-thought (CoT) prompting enhances large language model performance, yet what drives these gains remains unclear. We study this question from a probe-time perspective: holding CoT rationales fixed, we test which textual properties matter for the final prediction. Across multiple datasets and model configurations, we find that randomizing the order of rationale sentences has little effect on accuracy, suggesting that the global order of reasoning steps is not the main source of the probe-time benefit. Moreover, even when the words in a rationale are randomly reordered, performance remains well above the no-rationale baseline, indicating that the rationale's words remain useful even without their original order. Restoring only short-range word order further improves performance and brings it substantially closer to full CoT. In most settings, much of this local-order gain is already obtained with three-word windows. Control experiments rule out explicit answer copying, simple lexical cues, generic topical context, and general robustness to shuffling as the main explanations. Mechanistic analyses further show that short-window gains are largely formed in early-to-middle model layers, with answer-relevant evidence concentrated in local text spans. Together, these findings support a local co-occurrence activation (LCA) interpretation: the probe-time benefit of fixed rationales arises mainly from the words they contain and short-range word co-occurrences.

Artificial Intelligence (cs.AI)

arXiv:2605.26795 [cs.AI]

(or arXiv:2605.26795v2 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2605.26795 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.