aiqbench analysis
This paper proposes a semantic framework to rigorously evaluate AI outputs as engineered representations rather than factual assertions, distinguishing between justified claims and unsupported extrapolations. By defining specific failure modes like source mismatches and stale references, the work establishes a vocabulary for verifying the authority behind an AI's world-changing actions. This theoretical contribution is essential for building trustworthy systems where reliability depends on explicit justification rather than mere fluency.
Ceci n'est pas une pipe: AI systems as semantic abstractions
arXiv:2607.09489v2 Announce Type: replace Abstract: An AI system's output is not the fact or world state it appears to describe, but rather an engineered representation. We propose a semantic framework to describe AI systems, to be able to examine the correctness of such representations. To do so, we distinguish what is justified by accepted domain knowledge, what reference sources say, and what the system can currently use. This allows us to give precise definitions to common failures: extrapolation, refuted or unsupported assertion, sources versus knowledge mismatch, stale or refuted source, added hypotheses, unsupported use... We hope our framework gives a useful vocabulary for specifying and checking AI systems whose outputs, citations, tool calls, and world-changing actions must be justified by reliable claims and explicit authority rather than apparent fluency.
Abstract:An AI system's output is not the fact or world state it appears to describe, but rather an engineered representation. We propose a semantic framework to describe AI systems, to be able to examine the correctness of such representations. To do so, we distinguish what is justified by accepted domain knowledge, what reference sources say, and what the system can currently use. This allows us to give precise definitions to common failures: extrapolation, refuted or unsupported assertion, sources versus knowledge mismatch, stale or refuted source, added hypotheses, unsupported use... We hope our framework gives a useful vocabulary for specifying and checking AI systems whose outputs, citations, tool calls, and world-changing actions must be justified by reliable claims and explicit authority rather than apparent fluency.
Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
arXiv:2607.09489 [cs.AI]
(or arXiv:2607.09489v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2607.09489 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 219 words
Related coverage
A BIM-enabled, Agent-based Discrete-event Simulation Platform for Robotic Studies: A Method based on Graph Theory
arXiv · Jul 21, 2026
PIPE-Cypher: Automatic Enterprise Benchmark Generation for Text-to-Cypher Systems
arXiv · Jun 9, 2026
AI-Driven Framework for Adaptive Water Network Management with Proof-of-Concept Implementation: Addressing Non-Revenue Water in Jordan
arXiv · Jun 16, 2026
Scaling Karpathy’s LLM wiki: Why your knowledge base needs a graph
Neo4j · Aug 31, 2026