Observing Health Outcomes Using Remote Sensing Imagery and Geo-Context Guided Visual Transformer

Abstract:Visual transformers have driven major progress in remote sensing image analysis, particularly in object detection and segmentation. Recent vision-language and multimodal models further extend these capabilities by incorporating auxiliary information, including captions, question and answer pairs, and metadata, which broadens applications beyond conventional computer vision tasks. However, these models are typically optimized for semantic alignment between visual and textual content rather than geospatial understanding, and therefore are not well suited for representing or reasoning with structured geospatial layers. In this study, we propose Geo-Context Guided Visual Transformer that enhances remote sensing imagery processing with auxiliary geospatial guidance. The proposed approach introduces a geospatial embedding mechanism that converts heterogeneous geospatial variables into patches-aligned representations and an asymmetric geo-context guided attention module that uses structured geospatial context to modulate visual attention while preserving the input-centered representation stream. The module also assigns geospatial roles to attention heads, supporting structured interpretation of image-geospatial interactions. Experimental results show that the proposed framework outperforms pretrained remote-sensing vision-language models and graph-based spatial fusion baselines in disease prevalence prediction. Ablation and visualization analyses further indicate its value for health-related remote sensing tasks where comprehensive geospatial data may be limited, while providing interpretable spatial cues for subsequent public-health analysis.

21 pages, 6 figures, 3 tables. Revised version incorporating peer-review comments. Published in IEEE Transactions on Geoscience and Remote Sensing

Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

arXiv:2602.00110 [cs.CV]

(or arXiv:2602.00110v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2602.00110 Focus to learn more arXiv-issued DOI via DataCite

https://doi.org/10.1109/TGRS.2026.3725585 Focus to learn more DOI(s) linking to related resources

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.