analiza aiqbench
This research addresses a critical reliability gap in large language models by aligning answer confidence with the substantive quality of their chain-of-thought rationales. Using a GRPO-based reinforcement learning framework, the authors demonstrate that jointly optimizing for rationale support significantly reduces alignment errors across medical and mathematical benchmarks without sacrificing accuracy. The work underscores that trustworthy AI reasoning requires internal justifications to genuinely match model certainty levels rather than merely appearing plausible.
CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
arXiv:2606.14961v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning can improve LLM performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly supported. We study confidence--rationale alignment: whether a model's confidence in its committed answer is justified by its generated rationale. We introduce a GRPO-based reinforcement learning framework that jointly rewards answer correctness, committed-answer probability, and rubric-based rationale support, where the rubric assesses grounding, coherence, task match, and connection to the selected answer without revealing the gold answer to the judge. Across MedQA, MathQA, and OpenBookQA using three open-weight LLMs, our method reduces the confidence--rationale alignment error by up to 26.51% compared with untuned checkpoints, SFT, and correctness-only GRPO, while maintaining competitive accuracy and often improving calibration. These results show that reliable CoT reasoning requires not only confident answers, but rationales that substantively support them.
Abstract:Chain-of-thought (CoT) reasoning can improve LLM performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly supported. We study confidence--rationale alignment: whether a model's confidence in its committed answer is justified by its generated rationale. We introduce a GRPO-based reinforcement learning framework that jointly rewards answer correctness, committed-answer probability, and rubric-based rationale support, where the rubric assesses grounding, coherence, task match, and connection to the selected answer without revealing the gold answer to the judge. Across MedQA, MathQA, and OpenBookQA using three open-weight LLMs, our method reduces the confidence--rationale alignment error by up to 26.51% compared with untuned checkpoints, SFT, and correctness-only GRPO, while maintaining competitive accuracy and often improving calibration. These results show that reliable CoT reasoning requires not only confident answers, but rationales that substantively support them.
Computation and Language (cs.CL)
arXiv:2606.14961 [cs.CL]
(or arXiv:2606.14961v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.14961 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 157 cuvinte