aiqbench analysis
Code to Control advances real-time AI control by synthesizing Python policies that execute directly, eliminating the latency inherent in traditional LLM-based action selection or planning methods. By decoupling controller structure from parameters, this approach achieves faster decision speeds than standard PPO policies while maintaining strong performance across diverse simulation environments like Atari and MuJoCo. The technique represents a significant step toward practical, high-frequency AI agents capable of operating without continuous inference overhead.
Code to Control: Synthesizing Parameterized Reactive Controllers
arXiv:2609.38733v1 Announce Type: new Abstract: Recent LLM-based approaches to control either invoke a language model to select actions or synthesize world models that require planning at every decision, introducing latency that can limit real-time use. We introduce Code to Control, an approach that synthesizes Python controllers which execute directly as policies. Code to Control separates program structure from parameters. An LLM synthesizes the controller structure, while derivative-free search fits its parameters for continuous control using feedback from the environment. Once learned, the resulting controllers require neither LLM inference nor planning at decision time, enabling real-time gameplay and, under our timing protocol, faster action selection than a PPO policy. Across a suite of Atari games, Flappy Bird, and MuJoCo tasks, Code to Control outperforms planning-based program synthesis methods, remains competitive with deep reinforcement learning while using fewer environment interactions, transfers across substantial changes in environment dynamics, and scales to complex locomotion tasks.
Abstract:Recent LLM-based approaches to control either invoke a language model to select actions or synthesize world models that require planning at every decision, introducing latency that can limit real-time use. We introduce Code to Control, an approach that synthesizes Python controllers which execute directly as policies. Code to Control separates program structure from parameters. An LLM synthesizes the controller structure, while derivative-free search fits its parameters for continuous control using feedback from the environment. Once learned, the resulting controllers require neither LLM inference nor planning at decision time, enabling real-time gameplay and, under our timing protocol, faster action selection than a PPO policy. Across a suite of Atari games, Flappy Bird, and MuJoCo tasks, Code to Control outperforms planning-based program synthesis methods, remains competitive with deep reinforcement learning while using fewer environment interactions, transfers across substantial changes in environment dynamics, and scales to complex locomotion tasks.
17 pages. Code: this https URL
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.38733 [cs.AI]
(or arXiv:2609.38733v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2609.38733 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 249 words
Related coverage
Synthesizing Reactive Character Behaviors for Continuous Games via Programmatic Policy Search
arXiv · Sep 22, 2026
Article: Multi-Agent AI for Production Security Operations: an A2A and MCP Architecture in a 5G Core
InfoQ · Jul 23, 2026
Zero-Observation User Reactivation with Gap-Driven Dimensional Gating
arXiv · Jul 23, 2026
Closing the GPU Cluster Validation Gap: A Kubernetes-Native Approach with CVF
AMD · Jul 29, 2026