aiqbench analysis
This arXiv preprint proposes a generative pixel-based language model that reads text as rendered images instead of tokens, trained on eight high-resource languages and scaled to 0.5B parameters. Its main claims are better generalization to unseen low-resource scripts, a GRPO reinforcement-learning stage that reportedly replaces unstable adversarial training for free-running generation, and parity with tokenizer-based models on discriminative tasks. Readers should note that these results are self-reported at a small scale, and the central question is whether pixel-space generation can match token-based models on generative quality at larger sizes.
MIXAR: Scaling Autoregressive Pixel-based Language Models to Multiple Languages and Scripts
arXiv:2604.11575v2 Announce Type: replace Abstract: Pixel-based language models are gaining momentum as alternatives to traditional token-based approaches, promising to circumvent tokenization challenges; however, existing models remain English-centric and are hard to train as generators. This paper introduces MIXAR, the first generative multilingual pixel-based language model, pretrained on eight high-resource languages spanning Latin and CJK scripts and scaled up to 0.5B parameters. Although pretrained on high-resource languages only, MIXAR generalizes to unseen low-resource languages better than a tokenizer-based model pretrained on the same corpus, with its largest gains on the scripts most distant from the pretraining mix, taking a step toward more equitable language technology. To make free generation work in pixel space, MIXAR replaces the notoriously unstable adversarial training stage of prior pixel-based models with a GRPO-based reinforcement-learning stage that rewards the model on its own free-running generations: it proves more stable and multiplies free-generation accuracy by up to 2.3x on multilingual benchmarks. Finally, MIXAR performs on par with comparable tokenizer-based models on discriminative benchmarks, improves consistently with scale, and withstands orthographic attacks more gracefully than tokenizer-based models. Code and models are available at https://github.com/april-tools/mixar.
Abstract:Pixel-based language models are gaining momentum as alternatives to traditional token-based approaches, promising to circumvent tokenization challenges; however, existing models remain English-centric and are hard to train as generators. This paper introduces MIXAR, the first generative multilingual pixel-based language model, pretrained on eight high-resource languages spanning Latin and CJK scripts and scaled up to 0.5B parameters. Although pretrained on high-resource languages only, MIXAR generalizes to unseen low-resource languages better than a tokenizer-based model pretrained on the same corpus, with its largest gains on the scripts most distant from the pretraining mix, taking a step toward more equitable language technology. To make free generation work in pixel space, MIXAR replaces the notoriously unstable adversarial training stage of prior pixel-based models with a GRPO-based reinforcement-learning stage that rewards the model on its own free-running generations: it proves more stable and multiplies free-generation accuracy by up to 2.3x on multilingual benchmarks. Finally, MIXAR performs on par with comparable tokenizer-based models on discriminative benchmarks, improves consistently with scale, and withstands orthographic attacks more gracefully than tokenizer-based models. Code and models are available at this https URL.
29 pages. Code and models: this https URL
Computation and Language (cs.CL)
arXiv:2604.11575 [cs.CL]
(or arXiv:2604.11575v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2604.11575 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 283 words
Related coverage
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
The AI Inference Revolution Is Here
IEEE Spectrum · Sep 15, 2026
Seeing Without Understanding: Large Language Model Evaluation of Mobile User Interface Quality, Failure Taxonomy, and Architectural Explanation
arXiv · Sep 9, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026