From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations

Abstract:Large reasoning models generate reasoning trajectories token by token. At a given prefix, multiple next-token candidates may be plausible, yet shape subsequent reasoning and differ in their chances of leading to the correct answer. Whether these differences are reflected in internal representations remains unclear. To study this question, we define relative long-horizon candidate value to quantify within-prefix differences and construct the Reasoning Candidate-Token Value (RCTV) dataset, pairing mathematical reasoning prefixes with plausible next-token candidates and their empirical long-horizon candidate values. Using RCTV, we examine hidden states, multilayer perceptron outputs, and attention outputs at candidate positions with sparse linear probes. Experimental results provide evidence that relative long-horizon candidate value is linearly decodable from these representations at multiple layers. Candidate-value probes can also outperform probability- and embedding-based baselines. Furthermore, a controlled online intervention experiment shows that using the decoded signals to guide token selection during generation can improve the chances of eventual success. Our findings provide evidence that internal representations encode information about relative long-horizon candidate value, offering insights into how local candidate-level representations relate to long-horizon reasoning outcomes.

Computation and Language (cs.CL)

arXiv:2610.15412 [cs.CL]

(or arXiv:2610.15412v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.15412 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.