aiqbench analysis
This research advances AI robotics by moving beyond simple kinematic replication to simulate the rich multimodal sensorimotor experiences of infants on physical humanoids like iCub and virtual platforms such as pyCub. By reconstructing skeletal structures from video and mapping them onto these agents, the framework generates synthetic proprioceptive, tactile, and visual streams that enable deeper analysis of developmental milestones and neurodisorders. This work bridges computational modeling with embodied cognition to provide new tools for robotics and early detection systems in developmental science.
Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids
arXiv:2604.27583v2 Announce Type: replace-cross Abstract: Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at https://github.com/ctu-vras/motion-retargeting/.
Authors:Francisco M. López, Hoshinori Kanazawa, Ondrej Fiala, Yakov Balashov, Valentin Marcel, Lukas Rustler, Miles Lenz, Dongmin Kim, Yasuo Kuniyoshi, Jochen Triesch, Matej Hoffmann
Abstract:Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at this https URL.
Accepted at IEEE ICDL 2026. 8 pages, 6 figures. Cite as: F. M. López, H. Kanazawa, O. Fiala, Y. Balashov, V. Marcel, L. Rustler, M. Lenz, D. Kim, Y. Kuniyoshi, J. Triesch, and M. Hoffmann, "Simulating infant first-person sensorimotor experience via motion retargeting from babies to humanoids'', in 2026 IEEE International Conference on Development and Learning (ICDL). IEEE, 2026, pp. 1-8
Neurons and Cognition (q-bio.NC); Robotics (cs.RO)
arXiv:2604.27583 [q-bio.NC]
(or arXiv:2604.27583v2 [q-bio.NC] for this version)
From: Francisco M. López [view email] [v1] Thu, 30 Apr 2026 08:37:46 UTC (3,172 KB) [v2] Tue, 16 Jun 2026 10:46:16 UTC (3,172 KB)
Full text under the publisher's open licence · 303 words
Related coverage
Models Can't Remember Their Training. Neither Can You.
LessWrong · Jul 19, 2026
Infant simulator with an embodied caregiver: Generating infant-perspective touch and vision during social interaction
arXiv · Oct 6, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics
Latent Space · Oct 13, 2026