aiqbench analysis
This research introduces COA-Bench, a synthetic benchmark for evaluating adversarial multi-agent planning systems that generate and refine tactical courses of action through self-play. By quantifying doctrinal coherence and Nash-gap distances in zero-shot scenarios, the study provides a reproducible framework for assessing robustness against adversarial responses without relying on classified or operational data.
Adversarial Course-of-Action Generation: Game-Theoretic Multi-Agent Algorithms for COA matching & COA generation
arXiv:2609.26059v1 Announce Type: new Abstract: Course-of-action (COA) generation is a distributed planning problem: a system must propose structured candidate actions, evaluate them against an adversarial response, and surface options that remain tactically coherent under changing conditions. We present COA-Bench, a small offline benchmark and reproducibility artifact for comparing COA generation policies through self-play. Following the BattleCOA terminology, we reserve COA matching for asset-effect matching and COA generation for course-of-action generation; the present artifact does not implement either DecisionFunction directly. Instead, it represents COAs as typed action chains with conditional branches, assigns a synthetic COA quality score, compares opposing COAs with a BLUE-vs-RED advantage score and Nash-gap distance, and scores doctrinal coherence with an FM 3-0-inspired heuristic rubric. Across 50 synthetic scenarios spanning five operational templates, a sampled best-response policy that draws eight RED candidates reduces BLUE advantage from .516 to .485 and BLUE wargame win rate from .920 to .820; a two-stage multi-agent council with five BLUE proposer agents, RED-team adjudication, and critique-driven revision obtains .509 BLUE advantage and .820 BLUE win rate. We also identify and fix a benchmark-design issue in which scenario framing was stored as metadata but had no effect on generated COA content. COA-Bench is not an operational battle-management system and uses no real, classified, proprietary, or human-subject data. The contribution is an inspectable evaluation harness, preliminary benchmark evidence, and lessons for building auditable agentic planning artifacts.
Abstract:Course-of-action (COA) generation is a distributed planning problem: a system must propose structured candidate actions, evaluate them against an adversarial response, and surface options that remain tactically coherent under changing conditions. We present COA-Bench, a small offline benchmark and reproducibility artifact for comparing COA generation policies through self-play. Following the BattleCOA terminology, we reserve COA matching for asset-effect matching and COA generation for course-of-action generation; the present artifact does not implement either DecisionFunction directly. Instead, it represents COAs as typed action chains with conditional branches, assigns a synthetic COA quality score, compares opposing COAs with a BLUE-vs-RED advantage score and Nash-gap distance, and scores doctrinal coherence with an FM 3-0-inspired heuristic rubric. Across 50 synthetic scenarios spanning five operational templates, a sampled best-response policy that draws eight RED candidates reduces BLUE advantage from .516 to .485 and BLUE wargame win rate from .920 to .820; a two-stage multi-agent council with five BLUE proposer agents, RED-team adjudication, and critique-driven revision obtains .509 BLUE advantage and .820 BLUE win rate. We also identify and fix a benchmark-design issue in which scenario framing was stored as metadata but had no effect on generated COA content. COA-Bench is not an operational battle-management system and uses no real, classified, proprietary, or human-subject data. The contribution is an inspectable evaluation harness, preliminary benchmark evidence, and lessons for building auditable agentic planning artifacts.
5 pages. DAI 2026 Industry Track submission. Includes public reproducibility artifact link. Uses synthetic offline scenarios only; no classified data, proprietary data, human-subject data, or operational planning systems
Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
arXiv:2609.26059 [cs.AI]
(or arXiv:2609.26059v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.26059 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 275 words
Related coverage
Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · Sep 27, 2026
[AINews] not much happened today
Latent Space · Oct 3, 2026
Social Theory and Agent Foundations
LessWrong · Oct 8, 2026
Tariffs didn’t bring manufacturing jobs back to the US
The Verge · Jul 27, 2026