aiqbench analysis
MIL-BERT introduces a scalable neural architecture capable of classifying text segments exceeding one million tokens by leveraging multiple instance learning principles. This method achieves state-of-the-art performance on long-form datasets for tasks like political bias detection and author demographic inference, addressing critical encoding limits in current models. The approach also demonstrates strong generalization from weakly-labeled collections to individual instances, offering a robust solution for analyzing arbitrarily large textual corpora.
MIL-BERT: Classification of Arbitrarily Large Text with Performance and Explanatory Guarantees
arXiv:2608.20636v1 Announce Type: cross Abstract: Many text classification decisions are viable based on constituent excerpts alone. Taking inspiration from the field of multiple instance learning, we present an algorithm for training a neural network to classify text by selecting such excerpts. We show that our approach is also scalable with demonstrated learning against samples with nearly 1M tokens. We evaluate our methods on 7 datasets with emphasis on long-textual collections that far exceed the encoding limit of our base model. We present state-of-the-art results with this algorithm on 3 datasets: identification of political bias in news outlets, trigger warnings in long stories, and demographic characteristics of authors in tweet collections. Furthermore, the model trained on weakly-labeled collections of text (bags) generalizes to accurately classify constituent, smaller instances. Besides a new state-of-the-art for these problems, this approach is one of the few neural methods to excel in these datasets.
Abstract:Many text classification decisions are viable based on constituent excerpts alone. Taking inspiration from the field of multiple instance learning, we present an algorithm for training a neural network to classify text by selecting such excerpts. We show that our approach is also scalable with demonstrated learning against samples with nearly 1M tokens. We evaluate our methods on 7 datasets with emphasis on long-textual collections that far exceed the encoding limit of our base model. We present state-of-the-art results with this algorithm on 3 datasets: identification of political bias in news outlets, trigger warnings in long stories, and demographic characteristics of authors in tweet collections. Furthermore, the model trained on weakly-labeled collections of text (bags) generalizes to accurately classify constituent, smaller instances. Besides a new state-of-the-art for these problems, this approach is one of the few neural methods to excel in these datasets.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2608.20636 [cs.CL]
(or arXiv:2608.20636v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.20636 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 241 words
Related coverage
Clinical Domain Classification from Medical Transcriptions
arXiv · Sep 22, 2026
Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection
arXiv · Jul 21, 2026
Legal text classification in Korean sexual offense cases: from traditional machine learning to large language models with XAI insights
arXiv · Oct 2, 2026
Challenge: Hand coding weights for efficient sequence memorisation
AI Alignment Forum · Jul 23, 2026