aiqbench analysis
REALM addresses a critical bottleneck in supervised fine-tuning by jointly optimizing model parameters and annotator reliability scores without requiring prior knowledge of expert identity. This approach effectively filters out noise from heterogeneous crowdworker data, significantly improving performance on text classification and question answering benchmarks compared to standard majority voting or simple averaging techniques. By recovering annotator expertise unsupervised, the method ensures that large language models learn from higher-quality signals even when training data is inherently noisy.
REALM: Reliable Expertise-Aware Language Model Fine-Tuning from Noisy Annotations
arXiv:2604.17289v3 Announce Type: replace Abstract: Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise. Standard practice aggregates labels via majority vote or simple averaging, discarding annotator identity and causing the model to absorb the errors of unreliable annotators into its parameters. We propose REALM, which jointly learns the model parameters and a scalar expertise value for each annotator, entirely unsupervised and requiring nothing beyond annotator identity. The key idea is to model each observed label as a mixture between the model's prediction and a uniform random guess, weighted by the annotator's learned expertise. REALM applies to any task with a fixed label set, and extends to multiple tasks via a learned expertise matrix. On four text-classification datasets with \emph{real} crowdsourced annotations, REALM is the best method in all $12$ configurations of the three heterogeneous-annotator datasets, improving on the strongest applicable baseline, including majority vote and Dawid--Skene aggregation, by $+2.9$ points on average. On five question answering benchmarks with simulated noisy labels, it outperforms naive noisy fine-tuning in $152$ of $162$ configurations, by $+5.0$ points on average, with gains that grow with model capacity. The learned expertise additionally recovers annotator reliability without ever observing it. Our code is available at https://github.com/sajjad-ucsb/REALM
Abstract:Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise. Standard practice aggregates labels via majority vote or simple averaging, discarding annotator identity and causing the model to absorb the errors of unreliable annotators into its parameters. We propose REALM, which jointly learns the model parameters and a scalar expertise value for each annotator, entirely unsupervised and requiring nothing beyond annotator identity. The key idea is to model each observed label as a mixture between the model's prediction and a uniform random guess, weighted by the annotator's learned expertise. REALM applies to any task with a fixed label set, and extends to multiple tasks via a learned expertise matrix. On four text-classification datasets with \emph{real} crowdsourced annotations, REALM is the best method in all $12$ configurations of the three heterogeneous-annotator datasets, improving on the strongest applicable baseline, including majority vote and Dawid--Skene aggregation, by $+2.9$ points on average. On five question answering benchmarks with simulated noisy labels, it outperforms naive noisy fine-tuning in $152$ of $162$ configurations, by $+5.0$ points on average, with gains that grow with model capacity. The learned expertise additionally recovers annotator reliability without ever observing it. Our code is available at this https URL
Machine Learning (cs.LG)
arXiv:2604.17289 [cs.LG]
(or arXiv:2604.17289v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2604.17289 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 300 words
Related coverage
How to Fine-Tune Nemotron 3.5 ASR for Your Language, Domain, or Accent
Hugging Face · Jun 4, 2026
Conflict-Aware Federated Fine-Tuning of Large Language Models with Mixture-of-Experts
arXiv · Jun 16, 2026
Game-Theoretic Disempowerment: Loss of Control to Agencyless AI
LessWrong · Sep 27, 2026
Biological Superintelligence
LessWrong · Jul 30, 2026