analiza aiqbench
This research addresses a critical bottleneck in low-resource machine translation by demonstrating that data quality, specifically cleaning noise and orthographic variations, significantly boosts performance for LLM-based models in English-to-Belarusian tasks. The study provides a specialized pipeline that leverages large language model fine-tuning to overcome linguistic challenges unique to Belarusian, offering a practical framework for improving translation accuracy in underrepresented languages.
ReMova: Fine-tuning LLMs for English to Belarusian translation
arXiv:2609.16427v1 Announce Type: new Abstract: This paper presents a Belarusian-specific data-cleaning pipeline and fine-tuning for English-Belarusian machine translation. Our cleaning pipeline distinguishes itself from others by employing a correction tool that addresses the issue of the two orthographies of the Belarusian language, noise in the training data, interference from other languages and other misspelling issues common in Belarusian on the internet. A matched ablation on unfiltered training data shows substantial benefits from filtering for all fine-tuned models, with the LLM-based models gaining roughly twice as much from filtering as the dedicated encoder-decoder MT system, supporting the view that for Belarusian MT one of the primary bottlenecks is data quality.
Abstract:This paper presents a Belarusian-specific data-cleaning pipeline and fine-tuning for English-Belarusian machine translation. Our cleaning pipeline distinguishes itself from others by employing a correction tool that addresses the issue of the two orthographies of the Belarusian language, noise in the training data, interference from other languages and other misspelling issues common in Belarusian on the internet. A matched ablation on unfiltered training data shows substantial benefits from filtering for all fine-tuned models, with the LLM-based models gaining roughly twice as much from filtering as the dedicated encoder-decoder MT system, supporting the view that for Belarusian MT one of the primary bottlenecks is data quality.
Computation and Language (cs.CL)
arXiv:2609.16427 [cs.CL]
(or arXiv:2609.16427v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.16427 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 199 cuvinte
Subiecte conexe
Links #3: 2026/06 Part 1
LessWrong · 15 iun. 2026
Presentation: From Copy-Paste to Composition: Building Agents Like Real Software
InfoQ · 22 iul. 2026
Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following
arXiv · 24 sept. 2026
Multilingual Medical Reasoning for Question Answering with Large Language Models
arXiv · 2 sept. 2026