aiqbench analysis
MilliVid addresses the computational bottleneck of long-range consistency in video generation by introducing a coarse-to-fine rollout strategy within a multi-scale token space. This method compresses frames into hierarchical tokens, allowing the model to prioritize geometry and semantics at lower resolutions while reserving high-frequency details for later steps. The approach demonstrates improved object permanence and scene stability in extended sequences, such as Minecraft videos, without incurring the prohibitive sequence lengths typical of standard transformer architectures.
MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation
arXiv:2606.09056v1 Announce Type: cross Abstract: Video generative models have become increasingly powerful, but long-range consistency remains challenging to achieve because even a few dozen frames require impractically long transformer sequence lengths. We show that this issue can be mitigated by generating video using coarse-to-fine rollout within a multi-scale token space. Our approach is simple: first, we pre-train an autoencoder that compresses each frame into a hierarchy of tokens, with levels ranging from the typical latent resolution to only a handful of tokens per frame. The coarsest levels capture the most consequential information, such as scene layout and semantics, while finer levels add high-frequency appearance and texture. Then, we train a video diffusion model to generate these tokens using coarse-to-fine rollout. By carefully controlling the level of detail at which frames are generated and used as context during each rollout step, we are able to preserve long-range consistency in geometry and object permanence while spending less compute on the long-range consistency of less perceptually relevant details. We validate this approach using a custom dataset of long Minecraft videos, where it produces substantially more consistent rollouts compared to existing baselines.
Abstract:Video generative models have become increasingly powerful, but long-range consistency remains challenging to achieve because even a few dozen frames require impractically long transformer sequence lengths. We show that this issue can be mitigated by generating video using coarse-to-fine rollout within a multi-scale token space. Our approach is simple: first, we pre-train an autoencoder that compresses each frame into a hierarchy of tokens, with levels ranging from the typical latent resolution to only a handful of tokens per frame. The coarsest levels capture the most consequential information, such as scene layout and semantics, while finer levels add high-frequency appearance and texture. Then, we train a video diffusion model to generate these tokens using coarse-to-fine rollout. By carefully controlling the level of detail at which frames are generated and used as context during each rollout step, we are able to preserve long-range consistency in geometry and object permanence while spending less compute on the long-range consistency of less perceptually relevant details. We validate this approach using a custom dataset of long Minecraft videos, where it produces substantially more consistent rollouts compared to existing baselines.
Ishaan Preetam Chandratreya and David Charatan contributed equally. Project page: this https URL
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2606.09056 [cs.CV]
(or arXiv:2606.09056v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.09056 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 296 words
Related coverage
Can LLMs Follow Medical Expert Logic? A Benchmark for Hierarchical Logical Consistency in Risk-of-Bias Assessment
arXiv · Sep 12, 2026
Beyond Flat Labels: Level-Restricted Contrastive Learning for Hierarchical Fine-Grained Vision Classification
arXiv · Sep 30, 2026
Hierarchical Forecast Reconciliation for Urban Rail Transit Demand Prediction under Operational Disruptions
arXiv · Jun 8, 2026
H$^2$R: Hierarchical Hint-Based Reward for Mathematical Reasoning via Reinforcement Learning
arXiv · Oct 30, 2026