Attention in Krylov Space: Transformer-Based Extrapolation of Lanczos Coefficients

arXiv:2601.07937v2 Announce Type: replace Abstract: The Universal Operator Growth Hypothesis formulates time evolution of operators through Lanczos coefficients. In practice, however, numerical instability and memory cost limit the number of coefficients that can be exactly computed. In response to these challenges, the standard approach relies on fitting early coefficients to asymptotic forms, but such procedures can miss subleading, history-dependent structures in the coefficients that subsequently affect reconstructed observables. In this work, we treat the Lanczos coefficients as a causal time sequence and introduce a transformer-based model to autoregressively predict future Lanczos coefficients from short prefixes. For classical and quantum chaotic systems, our model outperforms asymptotic fits in both coefficient extrapolation and physical observable reconstruction, and achieves an order-of-magnitude reduction in error. The model also accurately extrapolates coefficients in integrable regimes, where no universal asymptotic fit exists. Remarkably, our model transfers across system sizes: it can be trained on smaller systems and then be used to extrapolate coefficients on a larger system \emph{without retraining}. By probing the learned attention patterns and performing targeted attention ablations, we identify portions of the coefficient history that are most influential for accurate forecasts. Our results demonstrate that modern sequence models can serve as practical surrogates for probing operator dynamics deep in Krylov space, where brute-force Lanczos iteration can be computationally prohibitive.

Abstract:The Universal Operator Growth Hypothesis formulates time evolution of operators through Lanczos coefficients. In practice, however, numerical instability and memory cost limit the number of coefficients that can be exactly computed. In response to these challenges, the standard approach relies on fitting early coefficients to asymptotic forms, but such procedures can miss subleading, history-dependent structures in the coefficients that subsequently affect reconstructed observables. In this work, we treat the Lanczos coefficients as a causal time sequence and introduce a transformer-based model to autoregressively predict future Lanczos coefficients from short prefixes. For classical and quantum chaotic systems, our model outperforms asymptotic fits in both coefficient extrapolation and physical observable reconstruction, and achieves an order-of-magnitude reduction in error. The model also accurately extrapolates coefficients in integrable regimes, where no universal asymptotic fit exists. Remarkably, our model transfers across system sizes: it can be trained on smaller systems and then be used to extrapolate coefficients on a larger system \emph{without retraining}. By probing the learned attention patterns and performing targeted attention ablations, we identify portions of the coefficient history that are most influential for accurate forecasts. Our results demonstrate that modern sequence models can serve as practical surrogates for probing operator dynamics deep in Krylov space, where brute-force Lanczos iteration can be computationally prohibitive.

Quantum Physics (quant-ph); Statistical Mechanics (cond-mat.stat-mech)

arXiv:2601.07937 [quant-ph]

(or arXiv:2601.07937v2 [quant-ph] for this version)

https://doi.org/10.48550/arXiv.2601.07937 Focus to learn more arXiv-issued DOI via DataCite

Phys. Rev. B 114, 014319 (2026)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

Attention in Krylov Space: Transformer-Based Extrapolation of Lanczos Coefficients

Related coverage