aiqbench analysis
This research introduces Parallel Language Models (PLMs), a novel architecture that generates multiple consistent tokens in a single forward pass to significantly reduce inference latency compared to standard autoregressive methods. By training with a unified objective function, the model achieves performance comparable to or exceeding traditional baselines while offering substantial speedups across math, coding, and instruction-following tasks. This advancement directly addresses the critical bottleneck of high inference latency as language models continue to scale in size and complexity.
Parallel Language Models via Variational Decoding
arXiv:2610.26693v1 Announce Type: new Abstract: Autoregressive language models emit one token per forward pass. This causes high inference latency, in particular as models are scaled up for quality. In this paper, we introduce Parallel Language Models (PLMs), which generate mutually consistent tokens in one forward pass by routing dependencies through latent auxiliary variables. PLMs are trained using a variational bound that reduces to an autoregressive loss plus a parallel decoding loss. Empirically, PLMs are on a faster and better Pareto frontier than autoregressive and discrete diffusion baselines. Pretrained PLMs achieve near-autoregressive pretraining loss on model scales of 80M to 3B parameters, and speedup follows a power law in model size. When post-trained, our 1.3B model matches or exceeds an identically trained autoregressive model on math, coding, and instruction-following benchmarks, speeding up inference $2.3\text{-}5.7\times$. Together, we show that parallel generation can be trained from a single training objective at autoregressive quality. We will make our code available at https://github.com/mandt-lab/plm.
Authors:Felix Draxler, Robin C. Geyer, Daniel Geyfman, Farrin Marouf Sofian, Justus Will, Philipp Hildebrandt, Giosue Migliorini, Prakhar Srivastava, Gagan Sai Trinadh Duggaraju Peddiraju, Kushagra Pandey, Jorge da Silva Gonçalves, Elijah Tavares, Jan Niklas Groeneveld, Yang Meng, Julia E. Vogt, Stephan Mandt
Abstract:Autoregressive language models emit one token per forward pass. This causes high inference latency, in particular as models are scaled up for quality. In this paper, we introduce Parallel Language Models (PLMs), which generate mutually consistent tokens in one forward pass by routing dependencies through latent auxiliary variables. PLMs are trained using a variational bound that reduces to an autoregressive loss plus a parallel decoding loss. Empirically, PLMs are on a faster and better Pareto frontier than autoregressive and discrete diffusion baselines. Pretrained PLMs achieve near-autoregressive pretraining loss on model scales of 80M to 3B parameters, and speedup follows a power law in model size. When post-trained, our 1.3B model matches or exceeds an identically trained autoregressive model on math, coding, and instruction-following benchmarks, speeding up inference $2.3\text{-}5.7\times$. Together, we show that parallel generation can be trained from a single training objective at autoregressive quality. We will make our code available at this https URL.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2610.26693 [cs.LG]
(or arXiv:2610.26693v1 [cs.LG] for this version)
Full text under the publisher's open licence · 211 words
Related coverage
Scan-Parallel Variational Inference for event-driven Latent Jump Models
arXiv · Oct 12, 2026
A foundation model with multi-variate parallel attention to generate neuronal activity
arXiv · Sep 1, 2026
Parallel classical simulation of noisy shallow circuits: no quantum advantage in 1D
arXiv · Oct 1, 2026
Neural Diversity Regularizes Hallucinations in Language Models
arXiv · Aug 6, 2026