aiqbench analysis
This paper introduces ANNOTARES, a novel dataset designed to evaluate models on extracting logical structures like legal conditions and consequences from German statutory texts. By benchmarking rule-based systems against modern Transformer architectures including BERT variants and LLMs, the study demonstrates that deep learning approaches significantly outperform traditional methods in capturing complex syntactic patterns within legal language. The release of this resource aims to advance automated legal reasoning research by providing a standardized evaluation framework for domain-specific NLP tasks.
ANNOTARES: A Dataset for Extracting Logical Structures from German Statutory Texts
arXiv:2608.03898v1 Announce Type: new Abstract: The automatic structural analysis of legal texts is a cornerstone of legal technology, yet the extraction of their logical components remains a significant challenge. In this paper, we introduce the task of identifying and segmenting legal conditions (Tatbestand) and legal consequences (Rechtsfolge) within German statutory texts. To support this task, we present ANNOTARES (Annotations of Tatbestand-Rechtsfolge Sequences), a novel dataset comprising German law texts with span-level annotations. Spanning three distinct legal codes, the dataset is designed to evaluate both domain-specific performance and cross-statute generalizability. We benchmark diverse architectural approaches: a rule-based baseline, CRFs, BiLSTMs, BiLSTM-CRF, and modern Transformer-based models, including BERT variants and LLM-based methods. Our results demonstrate that BERT and LLM-based models achieve superior performance in capturing the complex syntactic structures of legal language. We release our dataset to facilitate further research in automated legal reasoning.
Abstract:The automatic structural analysis of legal texts is a cornerstone of legal technology, yet the extraction of their logical components remains a significant challenge. In this paper, we introduce the task of identifying and segmenting legal conditions (Tatbestand) and legal consequences (Rechtsfolge) within German statutory texts. To support this task, we present ANNOTARES (Annotations of Tatbestand-Rechtsfolge Sequences), a novel dataset comprising German law texts with span-level annotations. Spanning three distinct legal codes, the dataset is designed to evaluate both domain-specific performance and cross-statute generalizability. We benchmark diverse architectural approaches: a rule-based baseline, CRFs, BiLSTMs, BiLSTM-CRF, and modern Transformer-based models, including BERT variants and LLM-based methods. Our results demonstrate that BERT and LLM-based models achieve superior performance in capturing the complex syntactic structures of legal language. We release our dataset to facilitate further research in automated legal reasoning.
Computation and Language (cs.CL)
arXiv:2608.03898 [cs.CL]
(or arXiv:2608.03898v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.03898 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 232 words
Related coverage
Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation
arXiv · Jul 31, 2026

Presentation: From Thousands to One: Building LLM-Powered Selection Systems
InfoQ · Aug 17, 2026

News
Quantum Computing Report · Jun 7, 2026

Build an end-to-end RFI questionnaire workflow using Amazon Quick Automate
AWS · Sep 10, 2026