aiqbench analysis
This research addresses a critical flaw in legal AI retrieval by quantifying "temporal leakage," where models artificially inflate performance using future citation data unavailable at deployment time. By implementing strict temporal fences, the study demonstrates that zero-training methods can achieve near-state-of-the-art results while maintaining rigorous deployability standards across US and European jurisdictions. This work is essential for building trustworthy legal AI systems that avoid over-optimistic evaluations based on impossible knowledge of future precedents.
Fenced Citation-Context Retrieval for Case Law: Temporal Leakage and Degree Control Across Two Jurisdictions
arXiv:2607.17142v1 Announce Type: cross Abstract: Incoming citation context-the text with which later cases describe a case when citing it-is a strong signal for legal precedent retrieval, but it is evaluated without a temporal fence: at query time the target precedent has not yet been cited by future cases, so an unfenced method credits itself with evidence unavailable at deployment. Our central contribution is to measure how much: a temporal-admission decomposition that splits the naive relax-the-fence gain into genuine future-citation leakage, legitimate pre-query admission, and an index effect, reporting the phantom fraction-the share that is future evidence. We instantiate this as a controlled deployability audit of incoming-citation retrieval across two jurisdictions, CLERC (US federal, 1.84M documents) and ECtHR-PCR (European Court of Human Rights), adding a zero-training anchor channel that admits only citers dated before the query. On ECtHR-PCR only 14.9% of the naive gain is genuine future evidence; on CLERC the over-credit grows to +4.95 R@1000 as the fence is relaxed. Under the fence, the zero-training channel gains +16.1 R@1000 over BM25 on CLERC, clears a citation-degree control, and reaches a published R@1000 estimate similar to the strongest trained system on ECtHR-PCR (79.56 vs 79.39) at zero training.
Abstract:Incoming citation context-the text with which later cases describe a case when citing it-is a strong signal for legal precedent retrieval, but it is evaluated without a temporal fence: at query time the target precedent has not yet been cited by future cases, so an unfenced method credits itself with evidence unavailable at deployment. Our central contribution is to measure how much: a temporal-admission decomposition that splits the naive relax-the-fence gain into genuine future-citation leakage, legitimate pre-query admission, and an index effect, reporting the phantom fraction-the share that is future evidence. We instantiate this as a controlled deployability audit of incoming-citation retrieval across two jurisdictions, CLERC (US federal, 1.84M documents) and ECtHR-PCR (European Court of Human Rights), adding a zero-training anchor channel that admits only citers dated before the query. On ECtHR-PCR only 14.9% of the naive gain is genuine future evidence; on CLERC the over-credit grows to +4.95 R@1000 as the fence is relaxed. Under the fence, the zero-training channel gains +16.1 R@1000 over BM25 on CLERC, clears a citation-degree control, and reaches a published R@1000 estimate similar to the strongest trained system on ECtHR-PCR (79.56 vs 79.39) at zero training.
Information Retrieval (cs.IR); Computation and Language (cs.CL)
arXiv:2607.17142 [cs.IR]
(or arXiv:2607.17142v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2607.17142 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 290 words
Related coverage
ABAI at COLIEE 2026 Task 1: Multi-Stage Retrieval with GraphRAG-Enhanced Meta-Learning, and a Post-Hoc Study of the Cross-Validation-to-Test Gap
arXiv · Sep 24, 2026
DySink: Dynamic Frame Sinks for Autoregressive Long Video Generation
arXiv · Jun 9, 2026
Cracking the Data Science Case Study Interview
Analytics Vidhya · Jul 26, 2026
From Vector Search to Agentic RAG: Building an Enterprise Research Analyst with hipVS
AMD · Jul 15, 2026