aiqbench analysis
This research demonstrates how agentic search frameworks can significantly enhance the discovery of NASA's vast Earth observation datasets using natural language queries. By combining knowledge graphs with LLM reasoning and autonomous tool use, the proposed system achieves substantial gains in retrieval metrics over traditional baselines. The work highlights the practical synergy between supervised learning pipelines and zero-shot LLM capabilities for solving complex domain-specific search problems.
Bringing Agentic Search to Earth Observation Data Discovery
arXiv:2607.02387v2 Announce Type: replace-cross Abstract: NASA and its data centers hold thousands of geoscience datasets and tools like Worldview, Giovanni, the Science Discovery Engine, and Harmony. Finding the right one is hard even for domain experts. We present an agentic search framework for geoscience data discovery that takes a natural-language research query and returns matching datasets and tools. We demonstrate that, in the era of large language models, the latent value of knowledge graphs (KGs) can be substantially amplified through agentic search. From the NASA Earth Observation Knowledge Graph (NASA EO-KG) we derive NASA-EO-Bench, an open benchmark of 47k query-dataset pairs (21k task-based queries). A neural scorer fine-tuned on NASA-EO-Bench beats cosine and BM25 baselines. Further combining it with BM25 via score fusion raises both Recall@10 (R@10) and MRR to over 5x the unadapted cosine baseline. On top of this supervised pipeline, a zero-shot reranking stage lifts MRR by 16%, significant under a paired bootstrap, with no additional training, and autonomous web and arXiv tool use adds a further gain, showing that LLM reasoning is complementary to supervised retrieval.
Abstract:NASA and its data centers hold thousands of geoscience datasets and tools like Worldview, Giovanni, the Science Discovery Engine, and Harmony. Finding the right one is hard even for domain experts. We present an agentic search framework for geoscience data discovery that takes a natural-language research query and returns matching datasets and tools. We demonstrate that, in the era of large language models, the latent value of knowledge graphs (KGs) can be substantially amplified through agentic search. From the NASA Earth Observation Knowledge Graph (NASA EO-KG) we derive NASA-EO-Bench, an open benchmark of 47k query-dataset pairs (21k task-based queries). A neural scorer fine-tuned on NASA-EO-Bench beats cosine and BM25 baselines. Further combining it with BM25 via score fusion raises both Recall@10 (R@10) and MRR to over 5x the unadapted cosine baseline. On top of this supervised pipeline, a zero-shot reranking stage lifts MRR by 16%, significant under a paired bootstrap, with no additional training, and autonomous web and arXiv tool use adds a further gain, showing that LLM reasoning is complementary to supervised retrieval.
Accepted at CIKM 2026 (full research paper). v2: camera-ready version; LLM rerank model sweep extended from N=200 to N=600 test queries with paired-bootstrap significance tests; adds a fine-tuned cross-encoder (bge-reranker-v2-m3) as a supervised reranking baseline
Information Retrieval (cs.IR); Machine Learning (cs.LG)
arXiv:2607.02387 [cs.IR]
(or arXiv:2607.02387v2 [cs.IR] for this version)
Full text under the publisher's open licence · 223 words
Related coverage
Synthesis Superintelligence: from Semiconductors to Superconductors — Periodic Labs’ Liam Fedus and Ekin Dogus Cubuk
Latent Space · Oct 8, 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · Sep 14, 2026
Tariffs didn’t bring manufacturing jobs back to the US
The Verge · Jul 27, 2026
Planetary prediction engine: Automating global models via Earth AI
Google Research · Aug 27, 2026