analiza aiqbench
This study challenges the standard practice of discrete tokenization in Chinese language modeling by demonstrating that a vision encoder processing rasterized character images outperforms traditional index-based embeddings. The research reveals a 21% accuracy improvement and faster convergence, suggesting that transformer architectures can effectively leverage spatial coherence inherent in logographic scripts without relying on discrete tokens. These findings indicate that the advantage is specific to Chinese characters' visual density and radical structure rather than being a universal property of all writing systems.
One Image, No Tokens: A Controlled Study of Glyph-Based Chinese Language Modeling
arXiv:2607.03994v2 Announce Type: replace-cross Abstract: Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned. We challenge this representation, especially for Chinese, by replacing index-based token embeddings entirely with a single rasterized image of the character sequence, processed by a vision encoder composed of a shared ResNet and a shallow Vision Transformer. To isolate the role of input representation, we construct a dual-branch controlled framework in which both a Vision-based model and an index-based baseline share an identical decoder backbone, training objective, optimizer, and data curriculum. Any performance difference is therefore attributable to the input modality only. Across all tested decoder backbones, the Vision-based model consistently outperforms the baseline, reaching a peak accuracy of 0.429 versus 0.355 for the index-based baseline---a 21\% relative improvement---while converging in about half the number of training epochs. The advantage emerges especially within the first five epochs (under 21\% of total data) and persists under moderate character corruption: the corrupted Vision model matches the \emph{clean} index-based baseline. Ablation studies reveal that the advantage requires both spatially coherent input and a ViT encoder with 2D positional encodings. A cross-script comparison on English shows the advantage does not transfer directly to alphabetic writing systems, suggesting that the uniform visual density and radical structure of Chinese characters are enabling conditions. These findings suggest that transformers are more modality-agnostic than commonly assumed, and that discrete tokenization is not a fundamental requirement for Chinese language modeling.
Abstract:Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned. We challenge this representation, especially for Chinese, by replacing index-based token embeddings entirely with a single rasterized image of the character sequence, processed by a vision encoder composed of a shared ResNet and a shallow Vision Transformer. To isolate the role of input representation, we construct a dual-branch controlled framework in which both a Vision-based model and an index-based baseline share an identical decoder backbone, training objective, optimizer, and data curriculum. Any performance difference is therefore attributable to the input modality only. Across all tested decoder backbones, the Vision-based model consistently outperforms the baseline, reaching a peak accuracy of 0.429 versus 0.355 for the index-based baseline---a 21\% relative improvement---while converging in about half the number of training epochs. The advantage emerges especially within the first five epochs (under 21\% of total data) and persists under moderate character corruption: the corrupted Vision model matches the \emph{clean} index-based baseline. Ablation studies reveal that the advantage requires both spatially coherent input and a ViT encoder with 2D positional encodings. A cross-script comparison on English shows the advantage does not transfer directly to alphabetic writing systems, suggesting that the uniform visual density and radical structure of Chinese characters are enabling conditions. These findings suggest that transformers are more modality-agnostic than commonly assumed, and that discrete tokenization is not a fundamental requirement for Chinese language modeling.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2607.03994 [cs.CV]
(or arXiv:2607.03994v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2607.03994 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 341 cuvinte
Subiecte conexe
Links #5: 2026/07
LessWrong · 31 iul. 2026
[AINews] Quasi-Riemann-Hypothesis: OpenAI publishes 722 math papers solving 90 of the top 500 open math problems; “the most significant moment” in >100 years of mathematics
Latent Space · 7 oct. 2026
Not Deciphering The Voynich Manuscript
LessWrong · 11 oct. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026