aiqbench analysis
Kozuchi Agent demonstrates that a locally hosted, open-weight LLM can effectively automate software repair across multiple programming languages without fine-tuning. The system achieves top-tier rankings on SWE-bench benchmarks by leveraging deterministic tools and persistent state management rather than relying on proprietary models. This work highlights the viability of auditable, CI-integrated agents for industrial-scale bug resolution using standard hardware configurations.
Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair
arXiv:2608.15579v1 Announce Type: cross Abstract: Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long horizons, tool-use discipline, context persistence, heterogeneous clusters, and evaluation reuse. We present Kozuchi Agent, a language-agnostic open-weight repair agent and CI-operated evaluation pipeline. Explicit phases, persistent state, deterministic tools, a model-independent action interface, and cross-agent test-time selection make runs auditable and repeatable. With locally hosted Qwen3.5-27B, no fine-tuning, and TTS@8, Kozuchi resolves 374/500 SWE-bench Verified instances on the official evaluator. Unchanged on Multi-SWE-bench Java, the same 27-billion-parameter agent resolves 41/128 instances (32.03%), ranking first among strict open-weight submissions and fourth of 42 overall; on Python it ranks 12th of 135 and first among open-weight systems. Per-phase behavior remains within +/-5 percentage points across languages. Remaining failures mainly reflect semantic correctness, Java-specific harness issues, and selection errors. Across both tracks, results compare favorably with open/local peers by parameter count. Analysis of candidate diversity, selector regret, and patch reliability shows that the remaining gap is primarily semantic correctness and selection rather than edit formatting or proprietary-model access. Operationally, reusable CI stages reduce operator touch-points from five to one across heterogeneous internal clusters.
Abstract:Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long horizons, tool-use discipline, context persistence, heterogeneous clusters, and evaluation reuse. We present Kozuchi Agent, a language-agnostic open-weight repair agent and CI-operated evaluation pipeline. Explicit phases, persistent state, deterministic tools, a model-independent action interface, and cross-agent test-time selection make runs auditable and repeatable. With locally hosted Qwen3.5-27B, no fine-tuning, and TTS@8, Kozuchi resolves 374/500 SWE-bench Verified instances on the official evaluator. Unchanged on Multi-SWE-bench Java, the same 27-billion-parameter agent resolves 41/128 instances (32.03%), ranking first among strict open-weight submissions and fourth of 42 overall; on Python it ranks 12th of 135 and first among open-weight systems. Per-phase behavior remains within +/-5 percentage points across languages. Remaining failures mainly reflect semantic correctness, Java-specific harness issues, and selection errors. Across both tracks, results compare favorably with open/local peers by parameter count. Analysis of candidate diversity, selector regret, and patch reliability shows that the remaining gap is primarily semantic correctness and selection rather than edit formatting or proprietary-model access. Operationally, reusable CI stages reduce operator touch-points from five to one across heterogeneous internal clusters.
13 pages, 4 figures. Accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE '26), Industry Showcase track, Munich, Germany, October 12-16, 2026
Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Programming Languages (cs.PL)
D.2.5; I.2.7; I.2.11
arXiv:2608.15579 [cs.SE]
(or arXiv:2608.15579v1 [cs.SE] for this version)
https://doi.org/10.48550/arXiv.2608.15579 arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1145/3832783.3834531 DOI(s) linking to related resources
Full text under the publisher's open licence · 250 words
Related coverage
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · May 21, 2026
Use open weight models as your AI coding agent with Amazon Bedrock
AWS · Sep 23, 2026
Mastering Agentic Techniques: AI Agent Customization
NVIDIA · May 20, 2026
Senior SWE-Bench: Evaluating Coding Agents Like Senior Engineers
Snorkel AI Blog · Jul 16, 2026