analiza aiqbench
This research addresses the computational bottleneck in factuality reinforcement learning for question answering by replacing expensive neural verifiers with a lightweight corpus-based verification method called CorVer. By leveraging co-occurrence statistics from indexed corpora, the approach significantly accelerates training time while improving factual accuracy across multiple models and benchmarks. The work highlights a critical efficiency gain for scaling RL post-training without relying on heavy inference overheads typical of current verification pipelines.
Beyond Math and Code: Lightweight Corpus-Grounded Process Rewards for Factual Question Answering
arXiv:2605.29648v2 Announce Type: replace Abstract: Process supervision during reinforcement learning (RL) post-training matters in factual question answering (QA) because responses receiving positive response-level rewards can still contain sentence-level factual errors. Unlike math and code, factual QA lacks inexpensive programmatic checks, making reward computation a bottleneck. Existing methods rely on neural verifiers to score individual sentences, requiring extensive model inference as RL repeats these checks across many sampled responses at every update. We therefore introduce CorVer (Corpus Verify), a lightweight training-time approach to process supervision that derives sentence-level rewards from corpus co-occurrence statistics. A 0.5B extractor identifies subject-object pairs, indexed corpus queries supply their co-occurrence counts, and the resulting sentence rewards are assigned to the corresponding tokens for RL. On Qwen3-4B and Qwen3-8B, CorVer reduces mean complete training time by 5.5-10.4 times relative to the four factuality-RL baselines. Across the four models evaluated against these baselines, CorVer achieves the highest accuracy in 17 of 20 model-benchmark settings. CorVer outperforms the unmodified models in all 30 standard factual-QA settings (six models from three families across five benchmarks) and remains effective on two additional multi-hop QA datasets.
Authors:Shicheng Fan, Haochang Hao, Dehai Min, Weihao Liu, Hanrong Zhang, Lingwei Wei, Henry Peng Zou, Chengquan Guo, Jie Yang, Honghui Bao, Zhiwei Liu, Lu Cheng, Philip S. Yu
Abstract:Process supervision during reinforcement learning (RL) post-training matters in factual question answering (QA) because responses receiving positive response-level rewards can still contain sentence-level factual errors. Unlike math and code, factual QA lacks inexpensive programmatic checks, making reward computation a bottleneck. Existing methods rely on neural verifiers to score individual sentences, requiring extensive model inference as RL repeats these checks across many sampled responses at every update. We therefore introduce CorVer (Corpus Verify), a lightweight training-time approach to process supervision that derives sentence-level rewards from corpus co-occurrence statistics. A 0.5B extractor identifies subject-object pairs, indexed corpus queries supply their co-occurrence counts, and the resulting sentence rewards are assigned to the corresponding tokens for RL. On Qwen3-4B and Qwen3-8B, CorVer reduces mean complete training time by 5.5-10.4 times relative to the four factuality-RL baselines. Across the four models evaluated against these baselines, CorVer achieves the highest accuracy in 17 of 20 model-benchmark settings. CorVer outperforms the unmodified models in all 30 standard factual-QA settings (six models from three families across five benchmarks) and remains effective on two additional multi-hop QA datasets.
28 pages, 7 figures. Code: this https URL
Computation and Language (cs.CL)
arXiv:2605.29648 [cs.CL]
(or arXiv:2605.29648v2 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 227 cuvinte
Subiecte conexe
Academia is for Ambition — Alex Zhang, MIT
Latent Space · 2 oct. 2026
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · 8 oct. 2026
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · 24 iul. 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026