analiza aiqbench
NEXUS introduces a perceptive whole-body control framework that enables humanoid robots to adapt their posture and contacts based on local terrain during teleoperation, addressing the scarcity of paired motion data across different environments. By combining human commands with onboard sensory feedback through teacher-student learning, the system achieves zero-shot deployment on unseen terrains while outperforming existing controllers in tracking fidelity and behavioral coverage. This advancement represents a significant step toward robust, generalizable whole-body control for humanoid robots operating in complex, real-world settings.
NEXUS: Perceptive Whole-Body Control for Terrain-Adaptive Teleoperation
arXiv:2609.39000v1 Announce Type: new Abstract: Whole-body teleoperation requires a humanoid robot to reproduce a human operator's behavior even when their terrains differ. This demands that the robot perceive local terrain and adapt its posture and contacts accordingly, rather than copy the operator's motion frame by frame. However, paired motion data linking the same behaviors across flat ground and different terrains remain scarce, limiting supervision for learning terrain-adaptive control. To enable whole-body teleoperation across mismatched terrains, we introduce NEXUS, a perceptive whole-body control framework that combines human motion commands with onboard sensory feedback. We first develop a scalable terrain-aware adaptation algorithm that efficiently generates high-quality motion pairs across motions and terrains without per-motion or per-terrain tuning. Using a paired motion corpus totaling nearly 1,000 hours, we train a perceptive whole-body controller through teacher-student learning to reproduce commanded behaviors across terrains. Experiments demonstrate efficient, scalable generation of high-quality motion data and show that NEXUS combines broad behavioral coverage with terrain adaptability and tracking fidelity, outperforming existing whole-body controllers on the evaluated benchmarks. Zero-shot real-world deployment enables real-time whole-body teleoperation on diverse unseen terrains, further validating the generalization of our method. Project website: https://nexus-humanoid.github.io/
Abstract:Whole-body teleoperation requires a humanoid robot to reproduce a human operator's behavior even when their terrains differ. This demands that the robot perceive local terrain and adapt its posture and contacts accordingly, rather than copy the operator's motion frame by frame. However, paired motion data linking the same behaviors across flat ground and different terrains remain scarce, limiting supervision for learning terrain-adaptive control. To enable whole-body teleoperation across mismatched terrains, we introduce NEXUS, a perceptive whole-body control framework that combines human motion commands with onboard sensory feedback. We first develop a scalable terrain-aware adaptation algorithm that efficiently generates high-quality motion pairs across motions and terrains without per-motion or per-terrain tuning. Using a paired motion corpus totaling nearly 1,000 hours, we train a perceptive whole-body controller through teacher-student learning to reproduce commanded behaviors across terrains. Experiments demonstrate efficient, scalable generation of high-quality motion data and show that NEXUS combines broad behavioral coverage with terrain adaptability and tracking fidelity, outperforming existing whole-body controllers on the evaluated benchmarks. Zero-shot real-world deployment enables real-time whole-body teleoperation on diverse unseen terrains, further validating the generalization of our method. Project website: this https URL
Robotics (cs.RO)
arXiv:2609.39000 [cs.RO]
(or arXiv:2609.39000v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2609.39000 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 281 cuvinte
Subiecte conexe
P-BFM: Perceptive Behavioral Foundation Models for Humanoid Control via Unsupervised Reinforcement Learning
arXiv · 14 oct. 2026
Learning Terrain-Aware Whole-Body Control for Perceptive Legged Loco-Manipulation
arXiv · 4 sept. 2026
PGMT: Perceptive General Motion Tracking for Humanoid Robots
arXiv · 9 sept. 2026
Perceptive Behavior Foundation Model: Adapting Human Motion Priors to Robot-Centric Terrain
arXiv · 16 iun. 2026