Skip to content
aiqbench takeAI Safety

aiqbench analysis

This research proposes a formal domain-specific language to enforce AI-SDLC protocols, distinguishing between declared policy and structural mechanism to prevent autonomous agents from skipping critical steps. Validation on SWE-bench Verified demonstrates that executing these formally specified processes yields significant performance gains over standard prose instructions by converting silent failures into auditable events. As foundation models converge in capability, the durable engineering asset shifts toward executable, self-correcting workflows rather than static prompts alone.
Source coveringarXiv
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Specifying AI-SDLC Processes: A Protocol Language for Human-Agent Boundaries

arXiv2 min read

arXiv:2606.20615v3 Announce Type: replace Abstract: AI agents now act as first-class members of the software development lifecycle, but the instruments teams use to direct them enforce nothing: process encoded in prompts is flexible but unenforceable, while workflow formalisms are enforceable but do not model autonomous agents. We propose a domain-specific language for specifying AI-SDLC processes as protocols, with formal abstract syntax, well-formedness conditions, operational semantics, and enforcement invariants, organised around a separation of policy (declared intent) from mechanism (structural enforcement). We prove that any well-formed protocol maintains its invariants on every execution trace, closed under Kleene composition of orchestration loops: protocol steps cannot be skipped, by construction. A failure-rate analysis derives two consequences: structural enforcement bounds silent failure while converting the remainder into visible, audited stalls, and the benefit carries a capability floor. We validate the design in simulation and end-to-end on SWE-bench Verified, where an identical bug-fix methodology yields no improvement delivered as prose instructions, but a significant, replicated 14-22 point gain when executed as a validated, self-correcting process; an ablation attributes the gain to the executed process itself, and the capability floor appears where the model predicts it. As foundation models converge, the durable engineering asset is the formally specified, executable process.

Abstract:AI agents now act as first-class members of the software development lifecycle, but the instruments teams use to direct them enforce nothing: process encoded in prompts is flexible but unenforceable, while workflow formalisms are enforceable but do not model autonomous agents. We propose a domain-specific language for specifying AI-SDLC processes as protocols, with formal abstract syntax, well-formedness conditions, operational semantics, and enforcement invariants, organised around a separation of policy (declared intent) from mechanism (structural enforcement). We prove that any well-formed protocol maintains its invariants on every execution trace, closed under Kleene composition of orchestration loops: protocol steps cannot be skipped, by construction. A failure-rate analysis derives two consequences: structural enforcement bounds silent failure while converting the remainder into visible, audited stalls, and the benefit carries a capability floor. We validate the design in simulation and end-to-end on SWE-bench Verified, where an identical bug-fix methodology yields no improvement delivered as prose instructions, but a significant, replicated 14-22 point gain when executed as a validated, self-correcting process; an ablation attributes the gain to the executed process itself, and the capability floor appears where the model predicts it. As foundation models converge, the durable engineering asset is the formally specified, executable process.

v2: substantially revised - enforcement soundness theorem with Lean 4 mechanisation, restructured failure-rate analysis, expressiveness study, end-to-end evaluation on SWE-bench Verified. 30 pages. Artifact: this https URL. Under review at ACM TOSEM

Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Programming Languages (cs.PL); Software Engineering (cs.SE)

D.2.1; D.2.4; D.3.1; I.2.11

arXiv:2606.20615 [cs.AI]

(or arXiv:2606.20615v3 [cs.AI] for this version)

Full text under the publisher's open licence · 255 words

Related coverage