aiqbench analysis
This research characterizes the unique runtime behavior of Vision-Language-Action (VLA) models in batch-1 inference scenarios required for real-time robotic control, revealing critical tradeoffs between latency, accuracy, and energy consumption on edge hardware. The findings demonstrate that standard LLM serving designs are insufficient for embodied AI, as action tensor dimensionality and platform balance fundamentally shift whether a stage is memory- or compute-bound. These insights provide essential guidance for jointly optimizing VLA architectures, underlying hardware, and runtime policies to meet strict robotic control deadlines.
Beyond LLM Serving: Characterizing Vision-Language-Action Workloads for Embodied AI System Design
arXiv:2610.05062v1 Announce Type: cross Abstract: Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets an inference deadline, and overruns leave the robot acting on stale observations, reducing task success. Meeting this deadline motivates on-device or nearby edge execution, where a single robot requires batch-1 inference outside the design point of LLM serving systems. Although VLA architectures combine familiar vision-language, autoregressive, and diffusion-style components, their runtime behavior in this batch-1 control setting remains uncharacterized. We characterize four representative VLA models on an edge GPU server and two onboard SoCs, using single-inference profiling and 43,200 closed-loop episodes. Action tensor dimensionality determines whether a stage is memory- or compute-bound, platform balance can shift that bottleneck, and GPU frequency scaling yields a platform-dependent energy-latency sweet spot. In closed-loop operation, overlapping inference with action execution creates an accuracy-speed-energy tradeoff, and no configuration is Pareto-dominant across deployment SLOs. These results guide joint design of VLA model architectures, hardware, and runtime policies.
Abstract:Vision-language-action (VLA) models translate multimodal observations into low-level robot actions. During robot operation, each control period sets an inference deadline, and overruns leave the robot acting on stale observations, reducing task success. Meeting this deadline motivates on-device or nearby edge execution, where a single robot requires batch-1 inference outside the design point of LLM serving systems. Although VLA architectures combine familiar vision-language, autoregressive, and diffusion-style components, their runtime behavior in this batch-1 control setting remains uncharacterized. We characterize four representative VLA models on an edge GPU server and two onboard SoCs, using single-inference profiling and 43,200 closed-loop episodes. Action tensor dimensionality determines whether a stage is memory- or compute-bound, platform balance can shift that bottleneck, and GPU frequency scaling yields a platform-dependent energy-latency sweet spot. In closed-loop operation, overlapping inference with action execution creates an accuracy-speed-energy tradeoff, and no configuration is Pareto-dominant across deployment SLOs. These results guide joint design of VLA model architectures, hardware, and runtime policies.
To appear in the Proceedings of the 32nd ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '27). 17 pages, 14 figures
Hardware Architecture (cs.AR); Robotics (cs.RO)
arXiv:2610.05062 [cs.AR]
(or arXiv:2610.05062v1 [cs.AR] for this version)
https://doi.org/10.48550/arXiv.2610.05062 arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1145/3845814.3850089 DOI(s) linking to related resources
Full text under the publisher's open licence · 210 words
Related coverage
[AINews] The Future of Latent Space
Latent Space · Sep 25, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · Sep 12, 2026