aiqbench analysis
This cs.SE/cs.AI preprint proposes "Semantic Engineering," a method for adding missing intent to LLM-integrated programs through language-level annotations that feed automatic prompt construction in the Jac/MTP stack. The authors report a 5.46x reduction in integration code and equal or better task performance in most of their seven applications, with gains up to 1.83x, across three models and two benchmarks. For AI readers, the practical question is whether these gains hold outside the authors' own toolchain and benchmarks, and how much the approach depends on Jac-specific infrastructure.
Prompt Less, Smile More: Semantic Engineering in Lieu of Prompt Engineering
arXiv:2511.19427v2 Announce Type: replace-cross Abstract: AI-integrated applications increasingly rely on large language models (LLMs), but their intended behavior is not always fully specified by code. Meaning Typed Programming (MTP) automates prompt construction from program structure, yet selection criteria, output requirements, and application policies can remain unstated. We introduce Semantic Engineering, a methodology for identifying such missing intent and selectively adding and refining model guidance. We realize this methodology through Semantic Context Annotations (SemTexts), a dedicated language mechanism independent of documentation and schema metadata. SemTexts associate natural-language instructions with individual program entities and use declaration scopes to determine where they apply. Our implementation extends the Jac compiler and MTP runtime to preserve these associations and select applicable guidance during automatic prompt construction. We evaluate Semantic Engineering on seven AI-integrated applications and two standard benchmarks across three models. Across the applications, it achieves a 5.46-fold reduction in integration lines of code relative to Prompt Engineering while delivering equal or better task performance in most model-application settings, reaching up to 1.83 times the task performance of Prompt Engineering.
Abstract:AI-integrated applications increasingly rely on large language models (LLMs), but their intended behavior is not always fully specified by code. Meaning Typed Programming (MTP) automates prompt construction from program structure, yet selection criteria, output requirements, and application policies can remain unstated. We introduce Semantic Engineering, a methodology for identifying such missing intent and selectively adding and refining model guidance. We realize this methodology through Semantic Context Annotations (SemTexts), a dedicated language mechanism independent of documentation and schema metadata. SemTexts associate natural-language instructions with individual program entities and use declaration scopes to determine where they apply. Our implementation extends the Jac compiler and MTP runtime to preserve these associations and select applicable guidance during automatic prompt construction. We evaluate Semantic Engineering on seven AI-integrated applications and two standard benchmarks across three models. Across the applications, it achieves a 5.46-fold reduction in integration lines of code relative to Prompt Engineering while delivering equal or better task performance in most model-application settings, reaching up to 1.83 times the task performance of Prompt Engineering.
Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
arXiv:2511.19427 [cs.SE]
(or arXiv:2511.19427v2 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2511.19427 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 265 words
Related coverage
Stop Writing Your Semantic Model. Start Reviewing It.
AtScale · Oct 14, 2026
SoK: Semantic Decision Engines in Network Control Loops
arXiv · Oct 7, 2026
The Missing Knowledge Layer in Cognitive Architectures for AI Agents
arXiv · Jun 16, 2026
GUARD: Geometric Uncertainty-Aware Point Cloud Denoising and Segmentation for Robotic Hard Disk Drive Disassembly
arXiv · Oct 8, 2026