analiza aiqbench
This research addresses a critical gap in natural language processing by establishing foundational machine translation capabilities for Tangkhul, an extremely low-resource Tibeto-Burman language lacking prior digital infrastructure. By leveraging fine-tuned transformer models like ByT5 and mT5 on limited parallel data, the study demonstrates viable pathways to digitize linguistic heritage despite challenges with specific orthographic diacritics and domain bias in training corpora. The work underscores ongoing efforts to expand AI accessibility beyond major languages into linguistically diverse communities globally.
Neural Machine Translation for Low-Resource Tangkhul--English
arXiv:2606.25365v2 Announce Type: replace-cross Abstract: We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair. Tangkhul is a severely under-resourced Tibeto-Burman language spoken primarily in Manipur, India, with virtually no prior natural language processing infrastructure. We describe two systems: (1) a primary system based on ByT5-large fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, and (2) a contrastive system based on mT5-small fine-tuned on the same corpus. Our primary ByT5-large system achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET (wmt22-comet-da) of 0.7302 on a held-out test set of 3,856 sentences. We further discuss the orthographic challenges specific to Tangkhul's Latin-script diacritics, the domain bias of our training corpus (which comprises biblical text, stories, and conversational data), and avenues for future improvement through data diversification and domain adaptation.
Abstract:We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair. Tangkhul is a severely under-resourced Tibeto-Burman language spoken primarily in Manipur, India, with virtually no prior natural language processing infrastructure. We describe two systems: (1) a primary system based on ByT5-large fine-tuned on 38,336 Tangkhul-English parallel sentence pairs, and (2) a contrastive system based on mT5-small fine-tuned on the same corpus. Our primary ByT5-large system achieves a corpus BLEU score of 39.97, chrF++ of 58.07, BERTScore F1 of 0.8104, and COMET (wmt22-comet-da) of 0.7302 on a held-out test set of 3,856 sentences. We further discuss the orthographic challenges specific to Tangkhul's Latin-script diacritics, the domain bias of our training corpus (which comprises biblical text, stories, and conversational data), and avenues for future improvement through data diversification and domain adaptation.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.25365 [cs.CL]
(or arXiv:2606.25365v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.25365 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 229 cuvinte
Subiecte conexe
Statistical Machine Translation Systems of English-Pnar Language Pair: Some Insights of the Emperical Study
arXiv · 25 aug. 2026
MetaHOPE: A Metaphor-Oriented Evaluation Framework for Analysing MT and LLM Translation Errors
arXiv · 25 iul. 2026
COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages
arXiv · 25 sept. 2026
Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
arXiv · 4 aug. 2026