aiqbench analysis
This research introduces the Insertion Process (IP), a novel generative framework that learns adaptive insertion orders to overcome limitations in fixed-grid diffusion models. By formalizing bijective correspondences between trajectories and permutations, the method enables variable-length generation with improved modeling quality for tasks lacking canonical left-to-right structures. The approach demonstrates enhanced generalization across goal-conditioned planning and molecular string synthesis through permutation-based variational inference.
Variational Learning for Insertion-based Generation
arXiv:2606.02133v3 Announce Type: replace-cross Abstract: Non-monotonic sequence generation methods, such as masked diffusion models, provide a flexible alternative to left-to-right autoregressive modeling by allowing tokens to be generated in non-fixed and prescribed orders. Despite their practical advantages, most existing non-monotonic models are order-agnostic and rely on a fixed-length grid, limiting their ability to support variable-length generation and adaptive insertion order. In this work, we introduce a probabilistic framework for learning insertion order in variable-length insertion models. We formalize a bijective correspondence between insertion trajectories and permutations, which enables an exact reparameterization of the data likelihood as a sum over permutations. Building on this result, we propose the Insertion Process (IP), a stochastic generative model that jointly learns where to insert, what to insert, and when to terminate, trained via permutation-based variational inference. Unlike prior fixed-canvas approaches, IP natively supports variable-length generation and learns data-driven preferences over insertion orders. Experiments on goal-conditioned planning and molecular string generation demonstrate that learning insertion order improves both modeling quality and generalization in domains without a canonical left-to-right structure.
Abstract:Non-monotonic sequence generation methods, such as masked diffusion models, provide a flexible alternative to left-to-right autoregressive modeling by allowing tokens to be generated in non-fixed and prescribed orders. Despite their practical advantages, most existing non-monotonic models are order-agnostic and rely on a fixed-length grid, limiting their ability to support variable-length generation and adaptive insertion order. In this work, we introduce a probabilistic framework for learning insertion order in variable-length insertion models. We formalize a bijective correspondence between insertion trajectories and permutations, which enables an exact reparameterization of the data likelihood as a sum over permutations. Building on this result, we propose the Insertion Process (IP), a stochastic generative model that jointly learns where to insert, what to insert, and when to terminate, trained via permutation-based variational inference. Unlike prior fixed-canvas approaches, IP natively supports variable-length generation and learns data-driven preferences over insertion orders. Experiments on goal-conditioned planning and molecular string generation demonstrate that learning insertion order improves both modeling quality and generalization in domains without a canonical left-to-right structure.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2606.02133 [cs.LG]
(or arXiv:2606.02133v3 [cs.LG] for this version)
Full text under the publisher's open licence · 183 words
Related coverage
Variational Learning for Insertion-based Generation
arXiv · Jun 10, 2026
Game Design Creativity evals
LessWrong · Oct 12, 2026
The Machines that Make the Machines
NVIDIA · Oct 7, 2026
Task aware Dynamic Movement Primitives for failure detection and recovery in contact rich manipulation
arXiv · Sep 22, 2026