aiqbench analysis
This paper introduces SpecWM, a JEPA framework that uses spectral representation learning to explicitly guide world models on which physical quantities to preserve via target similarity kernels. Theoretical analysis establishes conditions for linear recoverability and invariance to nuisance information, while empirical results demonstrate significant improvements in control planning success across multiple benchmarks compared to existing baselines. By steering latent geometry through kernel selection, this approach offers a principled method for enhancing the utility of world models in robotic control tasks.
World Modeling through Spectral Alignment
arXiv:2610.15429v1 Announce Type: cross Abstract: How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those relationships in its embedding geometry. Different kernels provide different supervision, ranging from physical state information to a fully self-supervised temporal kernel requiring only the ordering of observations within trajectories. We characterize the global optima of the objective, establish conditions for linear recoverability and invariance to nuisance information, and connect temporal-kernel similarity to the behavioral successor measure under an idealized setting. Empirically, the choice of kernel strongly controls which physical quantities are recoverable from the representation. More importantly, temporal spectral alignment improves CEM planning success over LeWorldModel from 52% to 68% on OGBench Cube, 49% to 77% on OGBench Scene, and 29% to 51% on CALVIN. These results suggest that explicitly specifying representation geometry provides a simple way to steer what latent world models preserve for control.
Abstract:How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those relationships in its embedding geometry. Different kernels provide different supervision, ranging from physical state information to a fully self-supervised temporal kernel requiring only the ordering of observations within trajectories. We characterize the global optima of the objective, establish conditions for linear recoverability and invariance to nuisance information, and connect temporal-kernel similarity to the behavioral successor measure under an idealized setting. Empirically, the choice of kernel strongly controls which physical quantities are recoverable from the representation. More importantly, temporal spectral alignment improves CEM planning success over LeWorldModel from 52% to 68% on OGBench Cube, 49% to 77% on OGBench Scene, and 29% to 51% on CALVIN. These results suggest that explicitly specifying representation geometry provides a simple way to steer what latent world models preserve for control.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2610.15429 [cs.LG]
(or arXiv:2610.15429v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.15429 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
Time series generation with spectrally aligned latent flow matching
arXiv · Sep 21, 2026
Aligning Inductive Bias for Data-Efficient Generalization in State Space Models
arXiv · Oct 2, 2026
Adaptive Oscillatory-State Alignment for Time Series Forecasting
arXiv · Jun 15, 2026