When More Documents Hurt RAG: Measuring and Reducing Vector Search Dilution with Domain-Scoped Retrieval

arXiv:2606.11350v2 Announce Type: replace Abstract: Retrieval-augmented generation (RAG) degrades when applied to large, heterogeneous document collections. In RAG systems, dense similarity becomes less discriminative, causing top-$k$ retrieval to return semantically similar but contextually incorrect chunks. We refer to this failure mode as vector search dilution. Even under hybrid dense+sparse retrieval, we first observed it in a deployed Wyoming Department of Transportation (WYDOT) system; in a controlled experiment, growing the corpus from 54 to 1,128 documents cut precision@10 by 63% and recall of the answer-bearing document by 23%. Similar patterns were observed in six other DOTs. To diagnose this issue, we introduce MASDR-RAG (Metadata-Scoped Domain Retrieval for RAG) as a controlled testbed over 200 human-validated queries, five LLM backbones, six public benchmark corpora, and two index stacks. In MASDR-RAG, domain scoping is the load-bearing component. We found hard partitioning to be unsafe, whereas training-free soft scoping is safe when the scope structure is real. Furthermore, we can discover these scopes without hand-curated metadata. However, multi-agent orchestration triggers a precision-faithfulness paradox that survives format controls, prompt controls, and blind human evaluation. Based on these varied outcomes, our practical recommendation is simple: soft-scope first, then perform a single synthesis call.

Abstract:Retrieval-augmented generation (RAG) degrades when applied to large, heterogeneous document collections. In RAG systems, dense similarity becomes less discriminative, causing top-$k$ retrieval to return semantically similar but contextually incorrect chunks. We refer to this failure mode as vector search dilution. Even under hybrid dense+sparse retrieval, we first observed it in a deployed Wyoming Department of Transportation (WYDOT) system; in a controlled experiment, growing the corpus from 54 to 1,128 documents cut precision@10 by 63% and recall of the answer-bearing document by 23%. Similar patterns were observed in six other DOTs. To diagnose this issue, we introduce MASDR-RAG (Metadata-Scoped Domain Retrieval for RAG) as a controlled testbed over 200 human-validated queries, five LLM backbones, six public benchmark corpora, and two index stacks. In MASDR-RAG, domain scoping is the load-bearing component. We found hard partitioning to be unsafe, whereas training-free soft scoping is safe when the scope structure is real. Furthermore, we can discover these scopes without hand-curated metadata. However, multi-agent orchestration triggers a precision-faithfulness paradox that survives format controls, prompt controls, and blind human evaluation. Based on these varied outcomes, our practical recommendation is simple: soft-scope first, then perform a single synthesis call.

19 pages, 5 figures, 20 tables. Preprint under review at ACL ARR October

Computation and Language (cs.CL); Information Retrieval (cs.IR)

arXiv:2606.11350 [cs.CL]

(or arXiv:2606.11350v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.11350 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

analiza aiqbench

When More Documents Hurt RAG: Measuring and Reducing Vector Search Dilution with Domain-Scoped Retrieval

Subiecte conexe