Context Aware AI Assistant and AR Interface for Lunar Extravehicular Activity (EVA) Procedural Guidance

Abstract:As human space exploration returns to the Moon, astronauts need rapid access to procedural information during extravehicular activities (EVAs), where attention is divided across navigation, repair tasks, tool handling, and environmental risk. The challenge is not the absence of information, but surfacing the right information at the right moment. We present GAIN-AI (Guided Assistant for Intelligent Navigation), a context-aware AI assistant and minimal heads-up interface for procedural guidance in simulated lunar EVA. The system operates in two layers. The first grounds a large language model with structured context: EVA procedure documents, live telemetry data, and error-handling protocols encoded as JSON. The second restructures that output into three compact units for AR display: Goal, Task, and Verification. Evaluated on 111 synthetic EVA scenarios, the system scores 10.0/10 on nominal conditions and 8.15/10 on single-fault scenarios, with performance degrading on multi-fault and boundary-threshold cases.

4 pages, 4 figures, 1 table. Accepted to ACM SIGGRAPH 2026 Conference Posters, July 19-23, 2026, Los Angeles, California, USA

Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)

H.5.1; H.5.2; I.2.7

arXiv:2608.13589 [cs.HC]

(or arXiv:2608.13589v1 [cs.HC] for this version)

https://doi.org/10.48550/arXiv.2608.13589 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

https://doi.org/10.1145/3799825.3818743 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.