analiza aiqbench
This research investigates how human evaluators and open-source LLMs weigh citations during preference modeling for scientific question answering, revealing distinct patterns in citation diversity versus quantity. The findings offer critical insights for refining reward models and post-training pipelines to better align model behavior with the need for verifiable grounding sources. By quantifying these preferences, the study directly addresses a core challenge in reducing hallucinations and improving the reliability of next-generation AI systems.
On the Role of Citations in Preference Data
arXiv:2608.21376v1 Announce Type: cross Abstract: Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a means for users to verify the credibility of model outputs. Yet, it is unclear how humans and LLMs evaluate citations when comparing outputs, a process central to reward modeling and modern LLM post-training. This paper studies the role of citations in the preferences of human judges and four open-source LLMs within the context of scientific question answering, leveraging mixed effects models to investigate the influence of citations on pairwise judgments. Among our key findings are (1) that humans prefer more diverse citations but fewer overall, and (2) that LLMs show some citation-related preferences compared to humans, despite lacking access to the sources, but these preferences depend on the data and specific models. We further discuss the implications of our findings for preference data collection.
Abstract:Many NLP tasks require systems to provide attribution in their outputs--i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a means for users to verify the credibility of model outputs. Yet, it is unclear how humans and LLMs evaluate citations when comparing outputs, a process central to reward modeling and modern LLM post-training. This paper studies the role of citations in the preferences of human judges and four open-source LLMs within the context of scientific question answering, leveraging mixed effects models to investigate the influence of citations on pairwise judgments. Among our key findings are (1) that humans prefer more diverse citations but fewer overall, and (2) that LLMs show some citation-related preferences compared to humans, despite lacking access to the sources, but these preferences depend on the data and specific models. We further discuss the implications of our findings for preference data collection.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2608.21376 [cs.CL]
(or arXiv:2608.21376v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.21376 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 246 cuvinte
Subiecte conexe
Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · 27 sept. 2026
Open-sourcing AstaBrief, the fast report-generation model in Asta
Hugging Face · 2 oct. 2026
DuplexGen: Adaptive Synthesis of Human-AI Turn-Taking Dialogues
arXiv · 30 iul. 2026
Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling
arXiv · 5 iun. 2026