analiza aiqbench
This research introduces Padyam2Gadyam, a dataset designed to evaluate Large Language Models on the nuanced task of translating classical Telugu poetry into modern prose. The study reveals that while general-purpose LLMs outperform traditional machine translation systems, both approaches face significant challenges in generating and evaluating faithful prose translations across languages. This work highlights critical gaps in current AI capabilities when handling low-resource literary domains and complex stylistic constraints.
Translating Classical Poetry into Modern Prose
arXiv:2606.02806v3 Announce Type: replace Abstract: We built a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu classical poetry to contemporary Telugu and English prose, which we call Padyam2Gadyam. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 2 machine translation systems and 5 contemporary Large Language Models (LLMs) on their ability to do zero-shot poem-to-prose translation into Telugu and English using this dataset. Our results indicate that while the general purpose LLMs are better than the machine translation systems, there are systematic issues with the generation and evaluation of prose translation in both languages across systems.
Abstract:We built a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu classical poetry to contemporary Telugu and English prose, which we call Padyam2Gadyam. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 2 machine translation systems and 5 contemporary Large Language Models (LLMs) on their ability to do zero-shot poem-to-prose translation into Telugu and English using this dataset. Our results indicate that while the general purpose LLMs are better than the machine translation systems, there are systematic issues with the generation and evaluation of prose translation in both languages across systems.
Computation and Language (cs.CL)
arXiv:2606.02806 [cs.CL]
(or arXiv:2606.02806v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.02806 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 194 cuvinte
Subiecte conexe
Translating Classical Poetry into Modern Prose
arXiv · 3 iun. 2026
Beyond Poetry: Can Large Language Models Generate Classical Arabic Maqamat?
arXiv · 24 sept. 2026
System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5
arXiv · 11 iun. 2026
News Archive 2026
Quantum Computing Report · 25 iul. 2026