analiza aiqbench
RePAIR introduces a self-supervised architecture that synthesizes MAE, JEPA, and BERT principles to learn compact representations of sequential chess positions without relying on costly reinforcement learning. By masking and predicting latent states, the model enables the emergence of meaningful chess concepts in its embedding space and facilitates intuitive game analysis through trajectory observation. This approach offers a novel framework for reasoning about piece movements and game dynamics using purely unsupervised learning techniques.
RePAIR: Predictive Self-Supervised Representation Learning in Chess
arXiv:2606.11860v1 Announce Type: new Abstract: In this paper, we introduce Representation Prediction via Autoencoding using Iterative Refinement (RePAIR) - a novel self-supervised representation learning architecture that synthesizes Masked Autoencoders (MAE), Joint Embedding Predictive Architectures (JEPA), and Bidirectional Encoder Representations from Transformers (BERT). We demonstrate how it can be used to encode objects in sequential data like consecutive chess positions into compact yet meaningful representations. The basic principle of the architecture is to mask large portions of a sequence of latent states, similar to BERT and MAE. Then, we apply a lightweight Predictor to the latent representations that repairs gaps in the sequence in a lower-dimensional embedding space akin to JEPA. Our experiments in the domain of chess show that the Encoder refines the board representations such that meaningful chess concepts emerge clustered in the latent space. Furthermore, reconstructions of the masked board states show that the model is able to reason about the piece movements without relying on costly reinforcement learning methods. Lastly, we find that the resulting representation space allows for quick and intuitive dissections of chess games by observing the game path trajectories in this semantically rich space.
Abstract:In this paper, we introduce Representation Prediction via Autoencoding using Iterative Refinement (RePAIR) - a novel self-supervised representation learning architecture that synthesizes Masked Autoencoders (MAE), Joint Embedding Predictive Architectures (JEPA), and Bidirectional Encoder Representations from Transformers (BERT). We demonstrate how it can be used to encode objects in sequential data like consecutive chess positions into compact yet meaningful representations. The basic principle of the architecture is to mask large portions of a sequence of latent states, similar to BERT and MAE. Then, we apply a lightweight Predictor to the latent representations that repairs gaps in the sequence in a lower-dimensional embedding space akin to JEPA. Our experiments in the domain of chess show that the Encoder refines the board representations such that meaningful chess concepts emerge clustered in the latent space. Furthermore, reconstructions of the masked board states show that the model is able to reason about the piece movements without relying on costly reinforcement learning methods. Lastly, we find that the resulting representation space allows for quick and intuitive dissections of chess games by observing the game path trajectories in this semantically rich space.
Machine Learning (cs.LG)
arXiv:2606.11860 [cs.LG]
(or arXiv:2606.11860v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.11860 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 279 cuvinte
Subiecte conexe
Models Can't Remember Their Training. Neither Can You.
LessWrong · 19 iul. 2026
Frozen Flows Forget: Diagnosing and Restoring Lost Motion in a Latent-flow World Model
arXiv · 24 sept. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026
Momentum-Guided Semantic Forecasting (MoFore) for Self-Supervised Video Representation Learning
arXiv · 16 iun. 2026