Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrieval

Abstract:Semantic code search and clone detection are essential for software development, maintenance, and reuse. This paper evaluates the effectiveness, efficiency, and scalability of contemporary deep learning models for first-stage recall in large-scale code-to-code search engines. Benchmarking across multiple programming languages and datasets reveals critical limits in the precision and scalability of these models on Terabyte-scale source-code collections. We present LLM-based code normalisation and query-rewriting schemes that yield significant gains in precision for lower-performing models. Our results question the sustainability of resource-constrained deployment and the assumed robustness of current code-specialised LLMs across datasets. We conclude with actionable insights for building scalable, efficient code-retrieval systems.

15 pages, 4 figures. Accepted for publication in the Proceedings of the 27th International Conference on Web Information Systems Engineering (WISE 2026). Preliminary version (differs in formatting and minor revisions from the final camera-ready version). Source code and benchmark are available at this https URL

Software Engineering (cs.SE); Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)

H.3.3; D.2.13; I.2.7

arXiv:2606.27401 [cs.SE]

(or arXiv:2606.27401v2 [cs.SE] for this version)