aiqbench analysis
This research demonstrates that cross-lingual alignment scores based on English representations effectively predict performance in both classification and translation tasks, suggesting LLMs internally pivot through English regardless of the source or target language. The study introduces a new PMI-based metric to evaluate these correlations across diverse languages without dependency on specific target vocabularies. These findings offer practical insights for optimizing multilingual model architectures by leveraging existing English-centric alignment mechanisms.
Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicode{x2013}$ Is English Enough?
arXiv:2608.03446v1 Announce Type: new Abstract: Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language.
Title:Predicting Multilingual Classification and Translation Performance of LLMs with Cross-Lingual Alignment $\unicode{x2013}$ Is English Enough?
Abstract:Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with chrF. We find that CLA with English predicts translation quality comparably to or better than source-target CLA, providing new evidence that LLMs use English as an internal pivot language.
Computation and Language (cs.CL)
arXiv:2608.03446 [cs.CL]
(or arXiv:2608.03446v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.03446 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 273 words
Related coverage
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
arXiv · Jun 6, 2026
Macro: Enhancing Multilingual Counterfactual Explanations through Alignment-as-Preference Optimization
arXiv · Aug 27, 2026
The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space
arXiv · Sep 2, 2026
Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora
arXiv · Oct 10, 2026