analiza aiqbench
TechPulse take: The Padyam2Gadyam dataset provides a rigorous benchmark for evaluating Large Language Models on low-resource, historically complex tasks involving Telugu and English. Results reveal significant performance gaps across current models, highlighting the need for improved evaluation methodologies beyond standard metrics. This work underscores the limitations of contemporary MT approaches when applied to specialized literary translation scenarios.
Translating Classical Poetry into Modern Prose
arXiv:2606.02806v1 Announce Type: new Abstract: We introduce Padyam2Gadyam, a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu Classical Poetry to contemporary Telugu and English prose. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 5 contemporary Large Language Models (LLMs) on their ability to do poem-to-prose translation into Telugu and English. Our results indicate that while there are differences across LLMs, their overall performance leave a large room for improvement in both languages. Through qualitative analysis, we discuss the the capabilities and limitations of contemporary MT evaluation approaches for this task.
Abstract:We introduce Padyam2Gadyam, a dataset for the task of poem-to-prose translation from 13th-17th Century Telugu Classical Poetry to contemporary Telugu and English prose. The dataset consists of 600 poems and their human-verified Telugu and English prose translations. We evaluated 5 contemporary Large Language Models (LLMs) on their ability to do poem-to-prose translation into Telugu and English. Our results indicate that while there are differences across LLMs, their overall performance leave a large room for improvement in both languages. Through qualitative analysis, we discuss the the capabilities and limitations of contemporary MT evaluation approaches for this task.
Computation and Language (cs.CL)
arXiv:2606.02806 [cs.CL]
(or arXiv:2606.02806v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.02806 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 191 cuvinte
Subiecte conexe
Translating Classical Poetry into Modern Prose
arXiv · 3 sept. 2026
Beyond Poetry: Can Large Language Models Generate Classical Arabic Maqamat?
arXiv · 24 sept. 2026
System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5
arXiv · 11 iun. 2026
News Archive 2026
Quantum Computing Report · 25 iul. 2026