A Bethe-Ansatz Source of Early Token Correlations: A marginal-preserving construction for parallel proposals

Abstract:Parallel token proposals need a joint law, not merely accurate token marginals. We construct coordinate-Bethe-ansatz laws from the periodic XXZ chain and compare them with their maximum-entropy pairwise projections at identical one- and two-site moments. In an eight-site, three-excitation sector, symmetry makes the full probability-law difference one-dimensional, determined by a contiguous three-site moment $\delta$. Along the Bethe branch, this gap changes sign and vanishes exactly at $\Delta=(3+\sqrt3)/2$, where the eigenstate law becomes pairwise. A nonlinear two-token readout preserves separate token marginals yet exposes a distance $2|\delta|$; a direct three-token readout preserves all one- and two-token marginals yet exposes $4|\delta|$. At $\Delta=1.5$ these deterministic binary distances are $0.02654573$ and $0.05309145$. We give an exact target-alignment formula and prove a strict speculative accepted-prefix advantage throughout an unrestricted open neighborhood of a constructed target. These are finite-law existence results, not measured language-model or wall-clock gains.

6 pages, 2 tables. Ancillary files reproduce the numerical results

Computation and Language (cs.CL); Machine Learning (cs.LG)

arXiv:2610.16781 [cs.CL]

(or arXiv:2610.16781v2 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.16781 Focus to learn more arXiv-issued DOI via DataCite

high_precision_xxz.py

three_token_verification.json

verification.json

verify_alignment.py

verify_three_token.py

xxz_delta_sweep.json

xxz_delta_sweep.py

(3 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.