analiza aiqbench
This arXiv paper is an AI research contribution: it introduces HCR-Bench, a 1,093-instance benchmark with step-level human causal reasoning annotations, and HCR-Reasoner, a method that adds token-causality formalisms and cognitive-science findings to LLM reasoning. The reported gains in alignment with human causal judgments are relative to the authors' own benchmark and baselines, so readers should check how the benchmark was built and how well the results transfer to other tasks. The work is most useful to researchers evaluating whether LLMs reason causally in human-like ways, and to those interested in combining symbolic causal theory with LLM prompting or training.
Synergizing Large Language Models and Theory for Human-like Causal Reasoning
arXiv:2505.08750v3 Announce Type: replace Abstract: Performing causal reasoning akin to humans is fundamental to achieving strong artificial intelligence. However, existing studies on causal reasoning and large language models (LLMs) often underestimate the role of human causal cognition. Human causal reasoning is far more than recognizing causes and effects. Humans often factor in many other considerations when establishing a causal link between two events, such as norm violations, intentionality, morality, counterfactual scenarios, etc. To simulate such reasoning, it is intuitive to identify cause-effect pairs using token causality formalisms while modeling the factors that influence human causal judgment using findings from cognitive science experiments. To support the evaluation and modeling of human-like causal reasoning in LLMs, we introduce HCR-Bench, a challenging benchmark with 1,093 instances and detailed human causal reasoning step annotations, and HCR-Reasoner, an approach that systematically integrates theories of token causality and cognitive science into LLMs for human-like causal reasoning. Results suggest that HCR-Reasoner consistently improves LLMs' causal alignment with humans, and that theory-guided reasoning is highly effective.
Abstract:Performing causal reasoning akin to humans is fundamental to achieving strong artificial intelligence. However, existing studies on causal reasoning and large language models (LLMs) often underestimate the role of human causal cognition. Human causal reasoning is far more than recognizing causes and effects. Humans often factor in many other considerations when establishing a causal link between two events, such as norm violations, intentionality, morality, counterfactual scenarios, etc. To simulate such reasoning, it is intuitive to identify cause-effect pairs using token causality formalisms while modeling the factors that influence human causal judgment using findings from cognitive science experiments. To support the evaluation and modeling of human-like causal reasoning in LLMs, we introduce HCR-Bench, a challenging benchmark with 1,093 instances and detailed human causal reasoning step annotations, and HCR-Reasoner, an approach that systematically integrates theories of token causality and cognitive science into LLMs for human-like causal reasoning. Results suggest that HCR-Reasoner consistently improves LLMs' causal alignment with humans, and that theory-guided reasoning is highly effective.
Computation and Language (cs.CL)
arXiv:2505.08750 [cs.CL]
(or arXiv:2505.08750v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2505.08750 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 256 cuvinte
Subiecte conexe
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
Links #3: 2026/06 Part 1
LessWrong · 15 iun. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Latent reasoning architectures would likely undermine CoT, our strongest oversight tool
AI Alignment Forum · 23 sept. 2026