analiza aiqbench
This research introduces PhySA, a reinforcement learning framework that decouples aesthetic scoring from physical smoothness to mitigate jerky gaits in humanoid robots. By employing separate reward channels, the method successfully reduces joint acceleration by up to 32% on MuJoCo simulations, though it does not currently generalize across different simulators or hardware platforms like the Unitree G1. The study highlights a critical trade-off where improved physical smoothness comes at the cost of shorter episode durations.
PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion
arXiv:2611.07679v1 Announce Type: new Abstract: Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.
Abstract:Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.
24 pages, 12 figures, 11 tables. Accepted to the CoRL 2026 Workshop on R2RL: Roadmap to Sample-Efficient Real-World Reinforcement Learning
Robotics (cs.RO)
arXiv:2611.07679 [cs.RO]
(or arXiv:2611.07679v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2611.07679 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 299 cuvinte
Subiecte conexe
RPL: Learning Robust Humanoid Perceptive Locomotion on Challenging Terrains
arXiv · 25 aug. 2026
DUET: Decoupled Upper- and Lower-Body Execution for Task-Agnostic Humanoid Loco-Manipulation
arXiv · 21 oct. 2026
PAINT: Partner-Agnostic Intent-Aware Cooperative Transport with Legged Robots
arXiv · 9 sept. 2026
PA-LOCO: Learning Perturbation-Adaptive Locomotion for Quadruped Robots
arXiv · 14 oct. 2026