aiqbench analysis
This research introduces a direct sign-to-sign translation architecture using Qwen3 to bypass intermediate text stages, effectively reducing error propagation between distinct signing communities like American, Chinese, and German Sign Language. By leveraging back-translation techniques to synthesize cross-lingual training pairs, the model achieves measurable improvements in motion accuracy and BLEU scores over traditional cascade methods. This advancement addresses a critical gap in facilitating seamless communication across diverse sign language users without relying on shared written languages.
Direct Translation between Sign Languages
arXiv:2605.20588v2 Announce Type: replace Abstract: Sign language translation has made substantial progress between sign and spoken languages, while translation across sign languages remains less explored. Translating directly between sign languages could support communication across signing communities without requiring a shared written language. A cascade of sign-to-text, spoken-language translation, and text-to-sign models offers one route, but can propagate intermediate errors and requires three sequential translation stages. We develop direct sign-to-sign translation, whose training is limited by the scarcity of parallel signing across languages. To address this obstacle, we adapt back-translation to construct cross-lingual pairs from existing text-sign corpora: the source signing is synthesized through a text bridge, while the target remains the gold sign from the original corpus. Using these pairs, we jointly train a single Qwen3-based model for text-to-sign and sign-to-sign translation. The latter generates target signing directly from source signing without an intermediate transcript. Experiments cover six directions among American, Chinese, and German Sign Language, using back-translation pairs and existing cross-lingual sign pairs. On synthetic sources, direct translation improves BLEU-4 in five directions and lowers overall motion error relative to the cascade. On existing test pairs, it improves BLEU-4 over the cascade by 1.91-3.87 points across all six directions.
Abstract:Sign language translation has made substantial progress between sign and spoken languages, while translation across sign languages remains less explored. Translating directly between sign languages could support communication across signing communities without requiring a shared written language. A cascade of sign-to-text, spoken-language translation, and text-to-sign models offers one route, but can propagate intermediate errors and requires three sequential translation stages. We develop direct sign-to-sign translation, whose training is limited by the scarcity of parallel signing across languages. To address this obstacle, we adapt back-translation to construct cross-lingual pairs from existing text-sign corpora: the source signing is synthesized through a text bridge, while the target remains the gold sign from the original corpus. Using these pairs, we jointly train a single Qwen3-based model for text-to-sign and sign-to-sign translation. The latter generates target signing directly from source signing without an intermediate transcript. Experiments cover six directions among American, Chinese, and German Sign Language, using back-translation pairs and existing cross-lingual sign pairs. On synthetic sources, direct translation improves BLEU-4 in five directions and lowers overall motion error relative to the cascade. On existing test pairs, it improves BLEU-4 over the cascade by 1.91-3.87 points across all six directions.
Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2605.20588 [cs.CL]
(or arXiv:2605.20588v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2605.20588 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 294 words
Related coverage
Putting sign language AI into users’ hands
Google DeepMind · Aug 12, 2026
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Simultaneous Translation between Sign Languages
arXiv · Sep 30, 2026