The functional and temporal roles of gaze evolve across the phases and constraints of multi-stage robot-mediated manipulation

Abstract:Goal-directed eye movements are a fundamental component of visuomotor control, enabling humans to anticipate and guide their actions. For this reason, they are increasingly used in human-robot interaction to estimate users' goals. However, during manipulation, fixations may reflect either an intended future action or the need to visually monitor the robotic proxy due to altered embodiment. How predictive and monitoring-related gaze are organized across the different phases of a constrained robot-mediated manipulation remains unclear. Here we address this question by investigating gaze behavior during goal-directed telemanipulation to characterize how visuomotor control adapts to altered embodiment in a multi-stage task. Our findings show that gaze remains strongly aligned with task goals, preserving its predictive role even during robot-mediated manipulation. At the same time, gaze frequently alternates between the robotic end-effector and the manipulated object, revealing increasing online monitoring. The presence and geometry of obstacles modulate the timing and distribution of these fixations, delaying attention to the final target when intermediate constraints become more demanding. These findings show that predictive gaze is not lost under altered embodiment but reorganized in response to changes in sensory feedback and control demands. More broadly, they highlight the flexibility of the human visuomotor system when the natural sensorimotor coupling is disrupted and suggest that gaze should be interpreted contextually rather than treating every fixation as direct evidence of user intention in human-robot interaction.

Robotics (cs.RO)

arXiv:2606.21920 [cs.RO]

(or arXiv:2606.21920v2 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2606.21920 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.