analiza aiqbench
This paper addresses a critical gap in NLP resources by introducing BaatCheet, a multilingual corpus designed specifically to train large language models on informal Indian language dialogues rather than formal text. By fine-tuning open-source LLMs on this dataset, the authors demonstrate significant improvements in capturing conversational nuances like speaker interaction and discourse coherence across five translation directions. This work directly advances the state of low-resource machine learning by providing essential training data for more naturalistic dialogue systems in Indic languages.
BaatCheet: A Multilingual Corpus for Dialogue Translation in Indian Languages
arXiv:2609.33296v1 Announce Type: new Abstract: Existing translation models are typically trained on sentence-level and formal text, limiting their ability to capture everyday conversational dialogue phenomena such as informality, speaker interaction, and discourse coherence. Most existing Indic translation resources and evaluation benchmarks focus on sentence-level or formal text, making it difficult to assess translation quality of the dialogue phenomena. In this work, we introduce BaatCheet, a multilingual dialogue corpus named after the Hindi term for conversation or chitchat, containing approximately 49,000 dialogues for dialogue translation across five translation directions. We fine-tune five open-source LLMs across seven training data configurations and find that fine-tuning yields substantial gains over zero- and few-shot baselines. To comprehensively evaluate dialogue translation quality, we employ multiple evaluation strategies, including automatic metrics, LLM-as-judge, and human assessments using an SQM-guided Direct Assessment (DA) Protocol.
Abstract:Existing translation models are typically trained on sentence-level and formal text, limiting their ability to capture everyday conversational dialogue phenomena such as informality, speaker interaction, and discourse coherence. Most existing Indic translation resources and evaluation benchmarks focus on sentence-level or formal text, making it difficult to assess translation quality of the dialogue phenomena. In this work, we introduce BaatCheet, a multilingual dialogue corpus named after the Hindi term for conversation or chitchat, containing approximately 49,000 dialogues for dialogue translation across five translation directions. We fine-tune five open-source LLMs across seven training data configurations and find that fine-tuning yields substantial gains over zero- and few-shot baselines. To comprehensively evaluate dialogue translation quality, we employ multiple evaluation strategies, including automatic metrics, LLM-as-judge, and human assessments using an SQM-guided Direct Assessment (DA) Protocol.
Computation and Language (cs.CL)
arXiv:2609.33296 [cs.CL]
(or arXiv:2609.33296v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.33296 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 225 cuvinte
Subiecte conexe
COILD: An Indic-Centric Parallel Corpus and Benchmark for Machine Translation Across Indian Languages
arXiv · 25 sept. 2026
BHARATI: Morphology-Aware Tokenizers for Classical Indian Languages with Subword Fertility Analysis
arXiv · 28 iul. 2026
Anuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages
arXiv · 15 oct. 2026
AgriGov: A Structured Multilingual Dataset Curation for Indian Government Schemes for Farmers
arXiv · 9 iun. 2026