analiza aiqbench
This research exposes a critical vulnerability in global LLM deployment where safety alignment achieved in English fails to transfer reliably to low-resource languages via translation. The study quantifies significant bypass rates across ten models and demonstrates that current evaluation metrics are unstable, fundamentally challenging the assumption of universal model safety.
Global Security Blind Spots: Translation as a Safety Bypass in Low-Resource Languages
arXiv:2610.27304v1 Announce Type: cross Abstract: Large language models are deployed globally, yet safety alignment and evaluation remain concentrated in English: harmful requests refused in English can elicit unsafe completions when translated into low-resource languages. We evaluate ten widely deployed LLMs across 79 languages (English plus 78 low-resource) under a translation-bypass threat model, using a metric conditioned on English safety, which separates translation bypass from unsafety already present in English. English safety does not transfer reliably: conditioned on safe English completions, mean bypass rates range from 21.9% to 44.4%. Bypass risk concentrates in a subset of language families, a pattern broadly shared across models. The shared ordering is confounded with translation degeneracy, but its variation across models is significant and survives every control we apply. We also show that per-model bypass rates do not identify a stable ranking: nine of ten models change rank across three conditioning schemes that estimate the same quantity, with no change to any underlying response.
Authors:Jalil Alva, David Harper, Abigail E. Thornton, Rafael S. Villalobos Alva, Leonardo Kenji Minemura Suazo, Konstantinos Karageorgos, Diego Velazquez, Sergio A. Serrano, Marvin Vargas Flores, Aaron Schliem, Fernando F. Migone
Abstract:Large language models are deployed globally, yet safety alignment and evaluation remain concentrated in English: harmful requests refused in English can elicit unsafe completions when translated into low-resource languages. We evaluate ten widely deployed LLMs across 79 languages (English plus 78 low-resource) under a translation-bypass threat model, using a metric conditioned on English safety, which separates translation bypass from unsafety already present in English. English safety does not transfer reliably: conditioned on safe English completions, mean bypass rates range from 21.9% to 44.4%. Bypass risk concentrates in a subset of language families, a pattern broadly shared across models. The shared ordering is confounded with translation degeneracy, but its variation across models is significant and survives every control we apply. We also show that per-model bypass rates do not identify a stable ranking: nine of ten models change rank across three conditioning schemes that estimate the same quantity, with no change to any underlying response.
19 pages, 4 figures. Accepted at NeurIPS 2026 Workshop: Foundations of Language Model Security (FLMSec)
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
arXiv:2610.27304 [cs.CL]
(or arXiv:2610.27304v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.27304 arXiv-issued DOI via DataCite (pending registration)
From: Jalil V. Alva [view email] [v1] Mon, 19 Oct 2026 21:20:49 UTC (1,128 KB)
Text integral sub licența deschisă a publicației · 243 cuvinte