aiqbench analysis
This research presents a context-aware multimodal fusion architecture for autonomous surface vessels (ASVs) that integrates RGB, infrared, and radar data to maintain reliable obstacle perception under varying environmental conditions like glare and darkness. The study demonstrates that this approach significantly outperforms rule-based or gated fusion methods in low-light scenarios while providing calibrated sensor status to navigation controllers. It represents a concrete advancement in edge AI for robotics, specifically addressing the robustness of perception systems when primary visual sensors fail.
Context-Aware RGB-Thermal-Radar Obstacle Perception for Resource-Constrained ASVs
arXiv:2610.23873v1 Announce Type: cross Abstract: Reliable obstacle perception is challenging for small autonomous surface vessels (ASVs): reflections and glare confound visible-light cameras, vessel motion shifts the apparent waterline, and darkness removes the dominant sensing modality altogether. Rather than treating multimodal fusion as a static accuracy gain, we propose a context-aware architecture that combines RGB and long-wave infrared (LWIR) imagery, frequency-modulated continuous-wave (FMCW) mmWave radar, and inertial context. A bearing-sector interface provides the navigation controller with calibrated obstacle probabilities, radar ranges, and sensor status. It also distinguishes sensor dropout from a valid observation with no detected obstacle. Using new multimodal lake recordings, we deployed the pipeline in a 24-leg closed-loop pilot under uncontrolled lake conditions (16 day, 4 dusk, and 4 night; 351 m), including night legs in which RGB was unusable and the vessel drove on LWIR and radar evidence alone. On eight held-out low-light recordings, the calibrated fusion model achieved average precision (AP) 0.869 and expected calibration error 0.108, compared with AP 0.673 for rule-based fusion, 0.620 for gated fusion, and 0.903 for RGB alone. Our results expose a central tradeoff: RGB ranks best when usable, whereas fusion preserves a navigation input when RGB fails. This work is a step toward perception that remains usable and degrades predictably across multi-day ASV missions.
Abstract:Reliable obstacle perception is challenging for small autonomous surface vessels (ASVs): reflections and glare confound visible-light cameras, vessel motion shifts the apparent waterline, and darkness removes the dominant sensing modality altogether. Rather than treating multimodal fusion as a static accuracy gain, we propose a context-aware architecture that combines RGB and long-wave infrared (LWIR) imagery, frequency-modulated continuous-wave (FMCW) mmWave radar, and inertial context. A bearing-sector interface provides the navigation controller with calibrated obstacle probabilities, radar ranges, and sensor status. It also distinguishes sensor dropout from a valid observation with no detected obstacle. Using new multimodal lake recordings, we deployed the pipeline in a 24-leg closed-loop pilot under uncontrolled lake conditions (16 day, 4 dusk, and 4 night; 351 m), including night legs in which RGB was unusable and the vessel drove on LWIR and radar evidence alone. On eight held-out low-light recordings, the calibrated fusion model achieved average precision (AP) 0.869 and expected calibration error 0.108, compared with AP 0.673 for rule-based fusion, 0.620 for gated fusion, and 0.903 for RGB alone. Our results expose a central tradeoff: RGB ranks best when usable, whereas fusion preserves a navigation input when RGB fails. This work is a step toward perception that remains usable and degrades predictably across multi-day ASV missions.
Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2610.23873 [cs.RO]
(or arXiv:2610.23873v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2610.23873 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 308 words