analiza aiqbench
TransBERT demonstrates that high-performance NLP models in specialized domains like life sciences can be built using exclusively synthetically translated data, effectively addressing the scarcity of non-English resources. This framework validates synthetic translation as a viable strategy for expanding language coverage without relying on parallel human-translated corpora. By releasing the TransCorpus toolkit and pre-trained model, the authors provide immediate assets for researchers tackling low-resource language pairs in technical fields.
TransBERT: A Framework for Synthetic Translation in Domain-Specific Language Modeling
arXiv:2609.26347v1 Announce Type: new Abstract: The scarcity of non-English language data in specialized domains significantly limits the development of effective Natural Language Processing (NLP) tools. We present TransBERT, a novel framework for pre-training language models using exclusively synthetically translated text, and introduce TransCorpus, a scalable translation toolkit. Focusing on the life sciences domain in French, our approach demonstrates that state-of-the-art performance on various downstream tasks can be achieved solely by leveraging synthetically translated data. We release the TransCorpus toolkit, the TransCorpus-bio-fr corpus (36.4GB of French life sciences text), TransBERT-bio-fr, its associated pre-trained language model and reproducible code for both pre-training and fine-tuning. Our results highlight the viability of synthetic translation in a high-resource translation direction for building high-quality NLP resources in low-resource language/domain pairs.
Abstract:The scarcity of non-English language data in specialized domains significantly limits the development of effective Natural Language Processing (NLP) tools. We present TransBERT, a novel framework for pre-training language models using exclusively synthetically translated text, and introduce TransCorpus, a scalable translation toolkit. Focusing on the life sciences domain in French, our approach demonstrates that state-of-the-art performance on various downstream tasks can be achieved solely by leveraging synthetically translated data. We release the TransCorpus toolkit, the TransCorpus-bio-fr corpus (36.4GB of French life sciences text), TransBERT-bio-fr, its associated pre-trained language model and reproducible code for both pre-training and fine-tuning. Our results highlight the viability of synthetic translation in a high-resource translation direction for building high-quality NLP resources in low-resource language/domain pairs.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.26347 [cs.CL]
(or arXiv:2609.26347v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.26347 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.18653/v1/2025.findings-emnlp.1053 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 230 cuvinte