aiqbench analysis
This paper introduces IsingFormer, a Transformer-based model that enhances Parallel Tempering by generating high-quality proposal moves for Markov Chain Monte Carlo sampling. By leveraging learned proposals from finite-temperature configurations, the method achieves significantly faster convergence on complex problems like 3D spin glasses and integer factorization without inflating the time-to-solution exponent. The work demonstrates a practical pathway for integrating generative AI directly into rigorous statistical physics workflows to accelerate optimization and sampling tasks.
IsingFormer: Augmenting Parallel Tempering With Learned Proposals
arXiv:2509.23043v2 Announce Type: replace-cross Abstract: Generative models have been extensively used to accelerate MCMC mixing for sampling and optimization, but their effective integration with standard MCMC remains an open question. Here, we introduce a global proposal move in which finite-temperature configurations from an external generator are used as proposals within Parallel Tempering (PT). We examine a specific generator, IsingFormer, a Transformer trained on long-run MCMC configurations intended to approximate equilibrium distributions, and call the resulting framework Transformer-Augmented Parallel Tempering (TAPT). The IsingFormer exhibits two useful capabilities: interpolation to untrained $\beta$ values and conditional completion under clamped settings absent from training. On 3D spin-glass instances, TAPT reaches substantially lower residual energies than standard PT in fewer Monte Carlo sweeps. On integer factorization, we show how a structured problem encoding can amortize the training cost by training IsingFormer once and reusing the same model across target products that were not imposed during training. Finally, in a scaling study that uses long-run MCMC configurations as proposals, with proposal generation excluded from the timing, TAPT reduces the fitted time-to-solution exponent by approximately $33\%$ relative to PT over the tested problem sizes.
Abstract:Generative models have been extensively used to accelerate MCMC mixing for sampling and optimization, but their effective integration with standard MCMC remains an open question. Here, we introduce a global proposal move in which finite-temperature configurations from an external generator are used as proposals within Parallel Tempering (PT). We examine a specific generator, IsingFormer, a Transformer trained on long-run MCMC configurations intended to approximate equilibrium distributions, and call the resulting framework Transformer-Augmented Parallel Tempering (TAPT). The IsingFormer exhibits two useful capabilities: interpolation to untrained $\beta$ values and conditional completion under clamped settings absent from training. On 3D spin-glass instances, TAPT reaches substantially lower residual energies than standard PT in fewer Monte Carlo sweeps. On integer factorization, we show how a structured problem encoding can amortize the training cost by training IsingFormer once and reusing the same model across target products that were not imposed during training. Finally, in a scaling study that uses long-run MCMC configurations as proposals, with proposal generation excluded from the timing, TAPT reduces the fitted time-to-solution exponent by approximately $33\%$ relative to PT over the tested problem sizes.
Machine Learning (cs.LG); Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI); Computational Physics (physics.comp-ph)
arXiv:2509.23043 [cs.LG]
(or arXiv:2509.23043v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2509.23043 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 283 words
Related coverage
CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agents
arXiv · Sep 22, 2026
Nonlocal Transition Kernel for Efficient Learning of Restricted Boltzmann Machines
arXiv · Aug 20, 2026
Combee: Scaling Prompt Learning for Self-Improving Language Model Agents
arXiv · Sep 28, 2026
BayesPO: Bayesian Prompt Optimization via Parallel-Tempered Gradient-Guided Discrete MCMC
arXiv · Jul 20, 2026