analiza aiqbench
This research demonstrates that Vision-Language Models can effectively generate informative captions for unlabeled astronomical images, enabling semantic search across over 100 million galaxy entries with zero-shot performance on rare phenomena. By combining VLM-generated descriptions with contrastive alignment and a re-ranking strategy, the approach significantly outperforms direct image similarity search and doubles recall for challenging targets. The work provides a scalable framework for unlocking large scientific archives in fields ranging from astrophysics to microscopy without requiring extensive manual labeling.
Semantic search for 100M+ galaxy images using AI-generated captions
arXiv:2512.11982v2 Announce Type: replace-cross Abstract: Finding scientifically interesting phenomena through slow manual labeling campaigns severely limits our ability to explore the billions of galaxy images produced by telescopes. In this work, we develop a pipeline to create a semantic search engine from completely unlabeled image data. Our method leverages Vision-Language Models (VLMs) to generate descriptions for galaxy images, then contrastively aligns a pre-trained astronomy foundation model with these embedded descriptions to produce searchable embeddings at scale. We find that current VLMs provide descriptions that are sufficiently informative to train a semantic search model that outperforms direct image similarity search. Our model, AION-Search, achieves state-of-the-art zero-shot performance on finding rare phenomena despite training on randomly selected images with no deliberate curation for rare cases. Furthermore, we introduce a VLM-based re-ranking method that nearly doubles the recall for our most challenging targets in the top-100 results. For the first time, AION-Search enables flexible semantic search for over 100 million galaxy images, enabling discovery from previously infeasible searches, including the identification of 36 new extragalactic stellar stream candidates. More broadly, our work provides an approach for making large, unlabeled scientific image archives semantically searchable, expanding data exploration capabilities in fields from Earth observation to microscopy. The code, data, and app are publicly available at https://github.com/NolanKoblischke/AION-Search
Abstract:Finding scientifically interesting phenomena through slow manual labeling campaigns severely limits our ability to explore the billions of galaxy images produced by telescopes. In this work, we develop a pipeline to create a semantic search engine from completely unlabeled image data. Our method leverages Vision-Language Models (VLMs) to generate descriptions for galaxy images, then contrastively aligns a pre-trained astronomy foundation model with these embedded descriptions to produce searchable embeddings at scale. We find that current VLMs provide descriptions that are sufficiently informative to train a semantic search model that outperforms direct image similarity search. Our model, AION-Search, achieves state-of-the-art zero-shot performance on finding rare phenomena despite training on randomly selected images with no deliberate curation for rare cases. Furthermore, we introduce a VLM-based re-ranking method that nearly doubles the recall for our most challenging targets in the top-100 results. For the first time, AION-Search enables flexible semantic search for over 100 million galaxy images, enabling discovery from previously infeasible searches, including the identification of 36 new extragalactic stellar stream candidates. More broadly, our work provides an approach for making large, unlabeled scientific image archives semantically searchable, expanding data exploration capabilities in fields from Earth observation to microscopy. The code, data, and app are publicly available at this https URL
Instrumentation and Methods for Astrophysics (astro-ph.IM); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2512.11982 [astro-ph.IM]
(or arXiv:2512.11982v2 [astro-ph.IM] for this version)
Text integral sub licența deschisă a publicației · 235 cuvinte
Subiecte conexe
From PDFs to insights: Architecting an intelligent document processing pipeline with AWS generative AI services
AWS · 12 iun. 2026
AI Search is now generally available
Cloudflare Blog · 1 oct. 2026
What Makes Linguistic Representations Good Models of High-Level Visual Perception in the Human Brain?
arXiv · 21 iul. 2026
Every Modality Through One API
OpenRouter · 16 iul. 2026