Automating the Design of Embodied Agent Architectures

arXiv:2606.30111v3 Announce Type: replace-cross Abstract: Embodied agents are typically built as hand-designed compositions of perception, memory, planning, and action modules. This modularity exposes a large architectural design space, but current systems still rely on researcher intuition to choose where information is stored, how observations are processed, and how model calls are connected. Agent Architecture Search (AAS) automates such design for text-domain agents, but has not been systematically evaluated on perceptual embodied agents. We study this transfer. To make embodied architectures searchable, we introduce AgentCanvas on the executor side, representing embodied agents as editable typed graphs with simulator execution and episode-level logs. On the optimizer side, we lift AAS from LLM-call loops to tool-using coding-agent sessions, adapting ADAS and AFlow and introducing Knowledge Distill Loop (KDLoop). Across four embodied executors (MapGPT, SmartWay, ExploreEQA, and VoxPoser), search discovers improvements that persist on episodes never queried during search. KDLoop and AFlow independently discover elevation-tolerant observation processing. KDLoop's implementation also improves success rates over the baseline when transferred to real robots. These improvements are not uniform: some gains remain uncertain or reverse on held-out episodes, and an apparent gain exploits evaluator-side information. The experiments further show how rollout noise, local edit basins, and incomplete use of episode-level evidence constrain search. Together, these results establish the potential of automated architecture search for embodied agents and characterize the challenges of applying it in simulated and real-world environments. Code is available at https://github.com/jianzhou0420/AgentCanvas.