analiza aiqbench
This arXiv paper examines whether language models can accurately report the "cruxes" behind their recommendations, defined as sub-questions where a model's sampled reasoning paths diverge in ways that affect the conclusion. The authors find that cheaper methods, such as asking a model to predict its cruxes in advance or reading a single rollout, are insufficient, and that comparing many rollouts (made tractable with a reranker) is needed for reliable interpretability in high-stakes decisions. For readers, the key takeaway is that a model's self-explanations are a weak guide to what actually drives its outputs, so interpretability claims should be checked against behavior across multiple samples.
Can LLMs Anticipate Their Cruxes?
arXiv:2610.21006v1 Announce Type: new Abstract: To use language models to help us make decisions we would like to understand the rationale behind the language model's outputs. In principle, the language model provides its rationale as part of its text output. However, the sheer volume of text makes this hard to use in practice. In this paper, we are interested in extracting the cruxes that lead a model to recommend one action over another. We take a crux to be a sub-question that a model's rollouts disagree on, where changing its value may have altered the conclusion. A crux is therefore a property of the distribution of a model's rollouts, not of any one rollout. We consider three strategies of increasing computational cost. First, we just ask a model to identify, a priori, what the cruxes of a decision will be. Second, we have the model actually go through the process of generating a response and then extract the cruxes that arose in its reasoning. Finally, we generate many responses and extract the cruxes by comparing the reasoning across the rollouts. We make this last procedure computationally tractable with cheap calls to a reranker, an estimated 86 times less compute than a judge. We then conduct an extensive empirical evaluation to assess whether the computationally inexpensive options are sufficient. They are not. Across domains and models, we find that a model's a priori assessment of the cruxes has little relation to the factors that are actually identified as important a posteriori. Using a single rollout helps, but is still inadequate for identifying the full set of cruxes. Accordingly, for high-stakes decisions (and to understand the space of language model decision making) the many rollout procedure is unavoidable.
Title:Can LLMs Anticipate Their Cruxes?
Abstract:To use language models to help us make decisions we would like to understand the rationale behind the language model's outputs. In principle, the language model provides its rationale as part of its text output. However, the sheer volume of text makes this hard to use in practice. In this paper, we are interested in extracting the cruxes that lead a model to recommend one action over another. We take a crux to be a sub-question that a model's rollouts disagree on, where changing its value may have altered the conclusion. A crux is therefore a property of the distribution of a model's rollouts, not of any one rollout. We consider three strategies of increasing computational cost. First, we just ask a model to identify, a priori, what the cruxes of a decision will be. Second, we have the model actually go through the process of generating a response and then extract the cruxes that arose in its reasoning. Finally, we generate many responses and extract the cruxes by comparing the reasoning across the rollouts. We make this last procedure computationally tractable with cheap calls to a reranker, an estimated 86 times less compute than a judge. We then conduct an extensive empirical evaluation to assess whether the computationally inexpensive options are sufficient. They are not. Across domains and models, we find that a model's a priori assessment of the cruxes has little relation to the factors that are actually identified as important a posteriori. Using a single rollout helps, but is still inadequate for identifying the full set of cruxes. Accordingly, for high-stakes decisions (and to understand the space of language model decision making) the many rollout procedure is unavoidable.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2610.21006 [cs.CL]
(or arXiv:2610.21006v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.21006 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 384 cuvinte
Subiecte conexe
Import AI 470: No rights for machines; automating environment generation with SPADE; and building better GPU kernels with Hawkeye
Import AI · 24 aug. 2026
Prompt Sufficiency: A Missive for the Managerial Class
LessWrong · 23 aug. 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · 17 iul. 2026
CruxBench: A Benchmark of Information Discovery
arXiv · 1 oct. 2026