aiqbench analysis
This study demonstrates that while Large Language Models can generate Entity-Relationship diagrams for simple requirements, their reliability degrades significantly as conceptual complexity increases due to inconsistencies and constraint failures. The findings suggest that current LLMs lack the maturity for autonomous use in complex database modeling without substantial validation overhead that negates productivity gains. Consequently, human oversight remains essential for robust data architecture tasks involving intricate logical structures.
On the Limitations of Large Language Models for Conceptual Database Modeling
arXiv:2605.11986v2 Announce Type: replace Abstract: This article analyzes the use of Large Language Models (LLMs) as support for the conceptual modeling of relational databases through the automatic generation of Entity-Relationship (ER) diagrams from natural language requirements. The approach combines different language models with prompt engineering techniques to evaluate their ability to identify entities, relationships, and attributes in a conceptually consistent manner. The experimental evaluation involved three LLMs, each subjected to three prompting techniques (Zero-Shot, Chain of Thought, and Chain of Thought + Verifier), applied to the same requirements scenario with progressively increasing complexity. The generated diagrams were qualitatively analyzed through direct comparison with the textual requirements, considering the structural and semantic adherence of the modeled elements. The results indicate that, although LLMs show reasonable performance in less complex scenarios, their reliability decreases as the complexity of the requirements increases, with a rise in inconsistencies, ambiguities, and failures in representing constraints. These findings reinforce that, in their current state, LLMs are not sufficiently mature for reliable use in complex scenarios, and the cost of validation may offset the apparent productivity gains.
Abstract:This article analyzes the use of Large Language Models (LLMs) as support for the conceptual modeling of relational databases through the automatic generation of Entity-Relationship (ER) diagrams from natural language requirements. The approach combines different language models with prompt engineering techniques to evaluate their ability to identify entities, relationships, and attributes in a conceptually consistent manner. The experimental evaluation involved three LLMs, each subjected to three prompting techniques (Zero-Shot, Chain of Thought, and Chain of Thought + Verifier), applied to the same requirements scenario with progressively increasing complexity. The generated diagrams were qualitatively analyzed through direct comparison with the textual requirements, considering the structural and semantic adherence of the modeled elements. The results indicate that, although LLMs show reasonable performance in less complex scenarios, their reliability decreases as the complexity of the requirements increases, with a rise in inconsistencies, ambiguities, and failures in representing constraints. These findings reinforce that, in their current state, LLMs are not sufficiently mature for reliable use in complex scenarios, and the cost of validation may offset the apparent productivity gains.
Artificial Intelligence (cs.AI)
arXiv:2605.11986 [cs.AI]
(or arXiv:2605.11986v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2605.11986 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 267 words
Related coverage
Angles of attack for continual learning safety
LessWrong · Jun 16, 2026
ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams
arXiv · Jul 28, 2026
Uncovering the Computational Ingredients of Human-Like Representations in LLMs
arXiv · Sep 2, 2026
Informe de riesgos de la IA de frontera (febrero–marzo de 2026)
METR · May 19, 2026