aiqbench analysis
This research proposes a formal domain-specific language to enforce AI-SDLC protocols, distinguishing between declared policy and structural mechanism to prevent autonomous agents from skipping critical steps. Validation on SWE-bench Verified demonstrates that executing these formally specified processes yields significant performance gains over standard prose instructions by converting silent failures into auditable events. As foundation models converge in capability, the durable engineering asset shifts toward executable, self-correcting workflows rather than static prompts alone.
Specifying AI-SDLC Processes: A Protocol Language for Human-Agent Boundaries
arXiv:2606.20615v3 Announce Type: replace Abstract: AI agents now act as first-class members of the software development lifecycle, but the instruments teams use to direct them enforce nothing: process encoded in prompts is flexible but unenforceable, while workflow formalisms are enforceable but do not model autonomous agents. We propose a domain-specific language for specifying AI-SDLC processes as protocols, with formal abstract syntax, well-formedness conditions, operational semantics, and enforcement invariants, organised around a separation of policy (declared intent) from mechanism (structural enforcement). We prove that any well-formed protocol maintains its invariants on every execution trace, closed under Kleene composition of orchestration loops: protocol steps cannot be skipped, by construction. A failure-rate analysis derives two consequences: structural enforcement bounds silent failure while converting the remainder into visible, audited stalls, and the benefit carries a capability floor. We validate the design in simulation and end-to-end on SWE-bench Verified, where an identical bug-fix methodology yields no improvement delivered as prose instructions, but a significant, replicated 14-22 point gain when executed as a validated, self-correcting process; an ablation attributes the gain to the executed process itself, and the capability floor appears where the model predicts it. As foundation models converge, the durable engineering asset is the formally specified, executable process.
Abstract:AI agents now act as first-class members of the software development lifecycle, but the instruments teams use to direct them enforce nothing: process encoded in prompts is flexible but unenforceable, while workflow formalisms are enforceable but do not model autonomous agents. We propose a domain-specific language for specifying AI-SDLC processes as protocols, with formal abstract syntax, well-formedness conditions, operational semantics, and enforcement invariants, organised around a separation of policy (declared intent) from mechanism (structural enforcement). We prove that any well-formed protocol maintains its invariants on every execution trace, closed under Kleene composition of orchestration loops: protocol steps cannot be skipped, by construction. A failure-rate analysis derives two consequences: structural enforcement bounds silent failure while converting the remainder into visible, audited stalls, and the benefit carries a capability floor. We validate the design in simulation and end-to-end on SWE-bench Verified, where an identical bug-fix methodology yields no improvement delivered as prose instructions, but a significant, replicated 14-22 point gain when executed as a validated, self-correcting process; an ablation attributes the gain to the executed process itself, and the capability floor appears where the model predicts it. As foundation models converge, the durable engineering asset is the formally specified, executable process.
v2: substantially revised - enforcement soundness theorem with Lean 4 mechanisation, restructured failure-rate analysis, expressiveness study, end-to-end evaluation on SWE-bench Verified. 30 pages. Artifact: this https URL. Under review at ACM TOSEM
Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Programming Languages (cs.PL); Software Engineering (cs.SE)
D.2.1; D.2.4; D.3.1; I.2.11
arXiv:2606.20615 [cs.AI]
(or arXiv:2606.20615v3 [cs.AI] for this version)
Full text under the publisher's open licence · 255 words
Related coverage
Building and evaluating model diffing agents
AI Alignment Forum · Jun 12, 2026

AI-powered metadata correction and harmonization
AWS · Aug 24, 2026

AgentOps: Operationalize agentic AI at scale with Amazon Bedrock AgentCore
AWS · Jun 1, 2026

Generate Autonomous Business Insights with AI Agent and MCP Servers
AWS · Jul 29, 2026