aiqbench analysis
This paper uses vision-language model embeddings to build semantic safety maps for robot navigation, and wraps them in conformal prediction to give calibrated coverage. The reported results are a 61.6% drop in semantic collisions on MP3D and 90.73% empirical coverage against a 90% target on a quadruped. The useful takeaway is the attempt to turn poorly calibrated VLM confidence into guarantees that hold as the hazard vocabulary grows and data drifts, though the guarantees and the real-world results are only as strong as the evaluation setup, which is limited to one dataset and one robot platform.
CoSMO: Conformal Semantic Mapping for Open-World Safe Robot Navigation
arXiv:2610.18321v1 Announce Type: new Abstract: Safe autonomous navigation requires not only geometric obstacle avoidance but also semantic awareness. Different object categories demand different safety clearances, and the set of relevant categories may expand as the robot explores novel environments. Vision-language models (VLMs) offer rich semantic representations for building maps of unknown scenes, yet their confidence scores are poorly calibrated, making raw predictions unreliable for safety-critical planning. Conformal prediction (CP) can provide coverage guarantees, but existing methods assume a fixed label set and stationary data, limiting their use in open-world settings. We propose Conformal Semantic Mapping for Open-World Safe Robot Navigation (CoSMO), a semantic safety mapping framework that transforms VLM embeddings into calibrated coverage sets and safety-aware obstacle dilations over a dynamically expanding hazard vocabulary, even under distribution shift. Experiments on the Matterport3D (MP3D) dataset show that CoSMO reduces semantic collisions by 61.6% relative to a geometry-only baseline, while maintaining formal coverage guarantees even under delayed expert feedback. Real-world experiments on a quadruped robot further demonstrate generalization by achieving long-run empirical coverage of 90.73% against the 90% target. Project website: https://yangming911.github.io/cosmo-page/.
Abstract:Safe autonomous navigation requires not only geometric obstacle avoidance but also semantic awareness. Different object categories demand different safety clearances, and the set of relevant categories may expand as the robot explores novel environments. Vision-language models (VLMs) offer rich semantic representations for building maps of unknown scenes, yet their confidence scores are poorly calibrated, making raw predictions unreliable for safety-critical planning. Conformal prediction (CP) can provide coverage guarantees, but existing methods assume a fixed label set and stationary data, limiting their use in open-world settings. We propose Conformal Semantic Mapping for Open-World Safe Robot Navigation (CoSMO), a semantic safety mapping framework that transforms VLM embeddings into calibrated coverage sets and safety-aware obstacle dilations over a dynamically expanding hazard vocabulary, even under distribution shift. Experiments on the Matterport3D (MP3D) dataset show that CoSMO reduces semantic collisions by 61.6% relative to a geometry-only baseline, while maintaining formal coverage guarantees even under delayed expert feedback. Real-world experiments on a quadruped robot further demonstrate generalization by achieving long-run empirical coverage of 90.73% against the 90% target. Project website: this https URL.
Robotics (cs.RO)
arXiv:2610.18321 [cs.RO]
(or arXiv:2610.18321v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.18321 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
Jev: System One models for Prod, not God — with Diogo Almeida, CEO, TypeSafe AI
Latent Space · Sep 21, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation
arXiv · Jul 21, 2026
Offloaded inference for real-world physical AI robotics
Microsoft Research · Sep 23, 2026