analiza aiqbench
This research tackles the critical challenge of label scarcity in domain-specific AI by integrating Retrieval-Augmented Generation (RAG) with fine-tuned detection models. The hybrid approach successfully enables zero-shot generalization across a 135-category taxonomy, significantly outperforming standalone BERT or RAG systems on hydropower licensing documents. For practitioners facing similar data sparsity issues in regulated industries, this demonstrates a viable path to deploying robust classification pipelines without exhaustive training data for every label.
Retrieval-Augmented Classification of Environmental Mitigations in Hydropower Licensing Documents
arXiv:2608.23241v1 Announce Type: cross Abstract: Identifying and classifying environmental mitigation obligations in Federal Energy Regulatory Commission hydropower licensing documents is a labor-intensive task requiring deep domain expertise. We formulate this as a multi-label classification problem over a structured 135-category taxonomy and address the central challenge of severe label scarcity: 40 of 135 categories have no training examples, and 26 have fewer than five. A supervised Bidirectional Encoder Representations from Transformers (BERT)-based pipeline, while effective on well-represented categories, achieves F1 of zero on unseen classes regardless of augmentation strategy. We introduce a Retrieval-Augmented Generation (RAG) pipeline that conditions classification on retrieved category definitions, enabling zero-shot generalization across the full label space. We further propose a hybrid system that combines BERT detection with RAG classification, exploiting the high recall of fine-tuned detection and the zero-shot coverage of retrieval-augmented reasoning. Evaluated on the full set of 2017 license documents (5,860 paragraphs, 135 categories), the hybrid achieves a Micro F1 of 0.524, outperforming the BERT-only pipeline (0.477) and the RAG-only pipeline (0.416) across all training-support buckets.
Abstract:Identifying and classifying environmental mitigation obligations in Federal Energy Regulatory Commission hydropower licensing documents is a labor-intensive task requiring deep domain expertise. We formulate this as a multi-label classification problem over a structured 135-category taxonomy and address the central challenge of severe label scarcity: 40 of 135 categories have no training examples, and 26 have fewer than five. A supervised Bidirectional Encoder Representations from Transformers (BERT)-based pipeline, while effective on well-represented categories, achieves F1 of zero on unseen classes regardless of augmentation strategy. We introduce a Retrieval-Augmented Generation (RAG) pipeline that conditions classification on retrieved category definitions, enabling zero-shot generalization across the full label space. We further propose a hybrid system that combines BERT detection with RAG classification, exploiting the high recall of fine-tuned detection and the zero-shot coverage of retrieval-augmented reasoning. Evaluated on the full set of 2017 license documents (5,860 paragraphs, 135 categories), the hybrid achieves a Micro F1 of 0.524, outperforming the BERT-only pipeline (0.477) and the RAG-only pipeline (0.416) across all training-support buckets.
Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
arXiv:2608.23241 [cs.IR]
(or arXiv:2608.23241v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2608.23241 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 264 cuvinte
Subiecte conexe
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · 24 sept. 2026
Improve contract search accuracy with auto-generated filters in Amazon Bedrock
AWS · 18 aug. 2026
Adapting AIM LLMs For Specific Use Cases Through Fine-Tuning in AMD AI Workbench
AMD · 3 iun. 2026
What is an AI Copilot?
Databricks · 28 aug. 2026