analiza aiqbench
This research introduces a self-evolution framework using Minimum Bayes Risk decoding and LLM-generated pseudo-labels to eliminate the need for expensive human annotations in Error Span Detection tasks. By outperforming supervised baselines trained on human data, the study demonstrates a viable path toward reducing reliance on manual labeling while maintaining high performance in machine translation evaluation.
Is Human Annotation Necessary? Iterative MBR Distillation for Error Span Detection in Machine Translation
arXiv:2603.12983v4 Announce Type: replace-cross Abstract: Error Span Detection (ESD) is a crucial subtask in Machine Translation (MT) evaluation, aiming to identify the location and severity of translation errors. While fine-tuning models on human-annotated data improves ESD performance, acquiring such data is expensive and prone to inconsistencies among annotators. To address this, we propose a novel self-evolution framework based on Minimum Bayes Risk (MBR) decoding, named Iterative MBR Distillation for ESD, which eliminates the reliance on human annotations by leveraging an off-the-shelf LLM to generate pseudo-labels. Extensive experiments on the WMT Metrics Shared Task datasets demonstrate that models trained solely on these self-generated pseudo-labels outperform both unadapted base model and supervised baselines trained on human annotations at the system and span levels, while maintaining competitive sentence-level performance.
Title:Is Human Annotation Necessary? Iterative MBR Distillation for Error Span Detection in Machine Translation
Abstract:Error Span Detection (ESD) is a crucial subtask in Machine Translation (MT) evaluation, aiming to identify the location and severity of translation errors. While fine-tuning models on human-annotated data improves ESD performance, acquiring such data is expensive and prone to inconsistencies among annotators. To address this, we propose a novel self-evolution framework based on Minimum Bayes Risk (MBR) decoding, named Iterative MBR Distillation for ESD, which eliminates the reliance on human annotations by leveraging an off-the-shelf LLM to generate pseudo-labels. Extensive experiments on the WMT Metrics Shared Task datasets demonstrate that models trained solely on these self-generated pseudo-labels outperform both unadapted base model and supervised baselines trained on human annotations at the system and span levels, while maintaining competitive sentence-level performance.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2603.12983 [cs.CL]
(or arXiv:2603.12983v4 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.12983 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 231 cuvinte
Subiecte conexe
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
Implementing and Evaluating a Basic Per-Action Monitor for Safer Evals
METR · 27 sept. 2026
Build a serverless PII redaction pipeline with Amazon Bedrock Data Automation
AWS · 16 sept. 2026
The Open ASR Leaderboard Adds Its First Global South Language
Hugging Face · 28 aug. 2026