analiza aiqbench
This study investigates whether visual aids like Euler diagrams enhance logical reasoning capabilities in contemporary large language models during syllogistic tasks. The findings indicate that while LLMs handle entailment and contradiction well, they do not consistently benefit from diagrammatic inputs and often commit systematic conversion errors on neutral problems. Consequently, the research suggests current architectures derive limited utility from visual representations for improving formal logic performance compared to standard text or logical notation alone.
Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
arXiv:2607.23513v1 Announce Type: cross Abstract: Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance. Recent work has also explored their effects on large language models (LLMs). In this paper, we compare four representational conditions for syllogistic reasoning: natural language, logical notation, linear diagrams, and Euler diagrams. Using 285 problems from Ando et al. (2024), we evaluate two contemporary LLMs, Claude 3.5~Sonnet and GPT-4o-mini. Our results show that diagrammatic representations do not consistently improve performance. Although the models perform well on entailment and contradiction problems, they struggle with neutral problems and often make systematic conversion errors. Overall, the results suggest that the tested models gain limited benefit from diagrams in logical reasoning tasks.
Title:Do Diagrams Help Large Language Models Reason? Evidence from Syllogistic Reasoning
Abstract:Diagrams are widely used to support logical reasoning, and prior studies suggest that representations such as Euler diagrams can improve human reasoning performance. Recent work has also explored their effects on large language models (LLMs). In this paper, we compare four representational conditions for syllogistic reasoning: natural language, logical notation, linear diagrams, and Euler diagrams. Using 285 problems from Ando et al. (2024), we evaluate two contemporary LLMs, Claude 3.5~Sonnet and GPT-4o-mini. Our results show that diagrammatic representations do not consistently improve performance. Although the models perform well on entailment and contradiction problems, they struggle with neutral problems and often make systematic conversion errors. Overall, the results suggest that the tested models gain limited benefit from diagrams in logical reasoning tasks.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2607.23513 [cs.CL]
(or arXiv:2607.23513v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.23513 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 230 cuvinte
Subiecte conexe
Article: Your Next DSL Author is a Language Model
InfoQ · 16 sept. 2026
Salesforce after Dreamforce: How the CRM giant can grow beyond its own interface
SiliconANGLE · 19 sept. 2026
A Candid Abacus AI Review: The All-in-One AI Platform for Professionals & Enterprises
KDnuggets · 10 sept. 2026
From Vector Search to Agentic RAG: Building an Enterprise Research Analyst with hipVS
AMD · 15 iul. 2026