aiqbench analysis
This research demonstrates that fine-tuning language models on the outputs of an optimal Bayesian model instills genuine probabilistic reasoning capabilities regarding hidden variables, a capability standard supervised fine-tuning on true answers fails to achieve. The study reveals that these "Bayesian-trained" models encode quantities of Bayes' rule within their internal representations and actively utilize these beliefs for decision-making under uncertainty. This finding highlights the critical importance of nuanced supervision strategies when developing LMs for tasks requiring robust inference from limited observations.
Bayesian Fine-tuning Yields Language Models that are as Bayesian as their Beliefs Allow
arXiv:2610.00679v1 Announce Type: new Abstract: Language models (LMs) are increasingly used for tasks that require reasoning about hidden variables from a few observations, for which Bayesian inference is the normatively correct solution. While supervised fine-tuning of an LM on the outputs of an optimal $\textit{Bayesian}$ model leads to near-Bayesian behavior, standard supervised fine-tuning (SFT) on the true answers for the task falls short of it. But behavior alone does not tell us $\textit{why}$ tuning on a $\textit{Bayesian}$ or an $\textit{oracle}$ (true answers) signal differs: whether the resulting LM represents Bayesian beliefs, acts on them, or turns them into a choice the way Bayes' rule does. To compare them, we formulate increasingly demanding requirements for an LM to count as a Bayesian decision maker, spanning its behavior, representations, and computations, and test them on a flight recommendation task. The Bayes-trained LM acts Bayesian, encodes quantities of Bayes' rule in its middle layers, and uses the encoded belief for the recommendation to a certain extent. The oracle-trained LM differs from it both in the beliefs it holds and whether it reads beliefs out into recommendations. Exchanging beliefs between the LMs transfers a part of the Bayesian advantage. Bayes fine-tuning thus installs usable Bayesian beliefs in an LM for reasoning under uncertainty in a way standard SFT on oracle answers cannot, highlighting the advantage of nuanced supervision.
Abstract:Language models (LMs) are increasingly used for tasks that require reasoning about hidden variables from a few observations, for which Bayesian inference is the normatively correct solution. While supervised fine-tuning of an LM on the outputs of an optimal $\textit{Bayesian}$ model leads to near-Bayesian behavior, standard supervised fine-tuning (SFT) on the true answers for the task falls short of it. But behavior alone does not tell us $\textit{why}$ tuning on a $\textit{Bayesian}$ or an $\textit{oracle}$ (true answers) signal differs: whether the resulting LM represents Bayesian beliefs, acts on them, or turns them into a choice the way Bayes' rule does. To compare them, we formulate increasingly demanding requirements for an LM to count as a Bayesian decision maker, spanning its behavior, representations, and computations, and test them on a flight recommendation task. The Bayes-trained LM acts Bayesian, encodes quantities of Bayes' rule in its middle layers, and uses the encoded belief for the recommendation to a certain extent. The oracle-trained LM differs from it both in the beliefs it holds and whether it reads beliefs out into recommendations. Exchanging beliefs between the LMs transfers a part of the Bayesian advantage. Bayes fine-tuning thus installs usable Bayesian beliefs in an LM for reasoning under uncertainty in a way standard SFT on oracle answers cannot, highlighting the advantage of nuanced supervision.
Computation and Language (cs.CL)
arXiv:2610.00679 [cs.CL]
(or arXiv:2610.00679v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.00679 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 314 words
Related coverage
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
DICE: Entropy-Regularized Equilibrium Selection for Stable Multi-Agent LLM Coordination
arXiv · Jun 9, 2026
BALAR: A Bayesian Agentic Loop for Active Reasoning
arXiv · Sep 30, 2026
Dynamic language model representations for multi-objective reaction optimisation
arXiv · Sep 11, 2026