aiqbench analysis
This research introduces CPR-Dataset, a large-scale benchmark designed to improve the retrieval of transferable scientific ideas across diverse research fields using LLM reranking and contrastive fine-tuning. The study demonstrates that standard retrievers struggle with cross-domain queries but can be significantly enhanced by training on this new dataset, which outperforms existing benchmarks in analogy and reasoning-intensive tasks.
Cross-Pollination Retrieval: Finding Transferable Ideas Across Fields in Scientific Literature
From arXiv
Full textarXiv:2610.25678v1 Announce Type: cross Abstract: Many scientific advances come from cross-pollination, carrying an idea from one field over to a problem in another. Yet retrieving papers across fields remains underexplored, with existing benchmarks confined to a narrow range of fields. We study cross-pollination retrieval, the problem of identifying transferable ideas across research fields, and introduce CPR-Dataset, a large-scale dataset constructed by mining cross-topic citations. Its evaluation split CPR-eval has 1,300 queries in 26 fields, and its training split CPR-train has 10,573 pairs.
On CPR-eval, the best of eleven existing retrievers reaches a Recall@10 of only 0.26, whereas an LLM reranker reaches 0.81. Standard contrastive fine-tuning on CPR-train raises Recall@10 to as high as 0.70. The gain carries over to domains held out from training and to analogy and reasoning-intensive retrieval. On AnaloBench, it raises nDCG@10 by up to 33% over the untrained retriever, whereas training on three existing retrieval datasets lowers it. On BRIGHT, it raises nDCG@10 by up to 70%, more than twice the gain from those datasets. CPR-Dataset is available at https://huggingface.co/CPR-Dataset, and the code at https://github.com/SeokwonJung-Jay/Cross-Pollination-Retrieval.
Abstract:Many scientific advances come from cross-pollination, carrying an idea from one field over to a problem in another. Yet retrieving papers across fields remains underexplored, with existing benchmarks confined to a narrow range of fields. We study cross-pollination retrieval, the problem of identifying transferable ideas across research fields, and introduce CPR-Dataset, a large-scale dataset constructed by mining cross-topic citations. Its evaluation split CPR-eval has 1,300 queries in 26 fields, and its training split CPR-train has 10,573 pairs.
On CPR-eval, the best of eleven existing retrievers reaches a Recall@10 of only 0.26, whereas an LLM reranker reaches 0.81. Standard contrastive fine-tuning on CPR-train raises Recall@10 to as high as 0.70. The gain carries over to domains held out from training and to analogy and reasoning-intensive retrieval. On AnaloBench, it raises nDCG@10 by up to 33% over the untrained retriever, whereas training on three existing retrieval datasets lowers it.
On BRIGHT, it raises nDCG@10 by up to 70%, more than twice the gain from those datasets. CPR-Dataset is available at this https URL, and the code at this https URL.
Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2610.25678 [cs.IR]
(or arXiv:2610.25678v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2610.25678 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 277 words