aiqbench analysis
The primary subject is a domain-specific LLM for historical reasoning, trained with a knowledge-graph-based factuality reward. It is squarely a machine learning research paper. The results are descriptive and based on LLM-judge ratings plus one benchmark, where the 7B model still trails the strongest baselines, so the claims are modest. The practical takeaway for AI readers is the idea of using entity and relation matching against reference knowledge as an RL reward to reduce factual drift in long reasoning chains.
Kongzi: A Historical Large Language Model with Fact Enhancement
arXiv:2504.09488v2 Announce Type: replace Abstract: Large Language Models (LLMs) can exhibit factual inaccuracies during long-chain reasoning or under misleading prompts, limiting their reliability in historical analysis. We propose Kongzi, a language model for historical understanding and reasoning that combines continued pre-training on historical corpora, supervised fine-tuning, and fact-enhanced reinforcement learning. Its factuality reward compares entities and relations extracted from generated responses with historical references through knowledge-graph similarity. We evaluate 0.5B and 7B models on historical question answering and counterfactual reasoning, using reported ratings from three LLM judges, and on CMMLU-Chinese-History. Kongzi-7B receives the highest historical-accuracy ratings among the compared models under all three judges. Reinforcement learning improves the mean answer scores of both model sizes and their CMMLU-Chinese-History accuracy, although Kongzi-7B remains below the strongest reported baselines on that benchmark. These descriptive results support further study of factuality rewards for historical language models.
Abstract:Large Language Models (LLMs) can exhibit factual inaccuracies during long-chain reasoning or under misleading prompts, limiting their reliability in historical analysis. We propose Kongzi, a language model for historical understanding and reasoning that combines continued pre-training on historical corpora, supervised fine-tuning, and fact-enhanced reinforcement learning. Its factuality reward compares entities and relations extracted from generated responses with historical references through knowledge-graph similarity. We evaluate 0.5B and 7B models on historical question answering and counterfactual reasoning, using reported ratings from three LLM judges, and on CMMLU-Chinese-History. Kongzi-7B receives the highest historical-accuracy ratings among the compared models under all three judges. Reinforcement learning improves the mean answer scores of both model sizes and their CMMLU-Chinese-History accuracy, although Kongzi-7B remains below the strongest reported baselines on that benchmark. These descriptive results support further study of factuality rewards for historical language models.
22 pages, 14 figures, 4 tables. Revised training and evaluation descriptions; corrected derived evaluation summaries and figures; updated author list
Computation and Language (cs.CL)
arXiv:2504.09488 [cs.CL]
(or arXiv:2504.09488v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2504.09488 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 251 words
Related coverage
XMatch: Enhancing Covariate-Aware Time Series Forecasting through Tree-Structured Exogenous Matching
arXiv · Sep 30, 2026
PILOT: Privileged Imitation Learning for End-to-End Motion Planning of Autonomous UAVs under Partial Observability
arXiv · Aug 17, 2026
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
arXiv · Jun 4, 2026
Semantics-Enhanced Retrieval-Augmented Time Series Forecasting
arXiv · Jun 16, 2026