GRPO-QM: Target Preserving Exploration for Quantum Tomography

arXiv:2609.14711v1 Announce Type: cross Abstract: Reward-based learning can alter the very posterior distribution that scientific inference aims to estimate. GRPO-QM sidesteps this by learning only an exploration strategy for a stated quantum-tomography posterior: a group-relative policy chooses among reversible physical moves, and an exact Metropolis correction ensures the posterior remains stationary once the policy is fixed. We then examine what learning contributes beyond physical proposal mechanisms and prior knowledge. Reconstruction comparisons suggest that most of the gains over the tested flows come from those two components rather than from learning itself, and a closed-form counterexample explains why: a reward tied to accepted motion can increase even when a physical observable remains highly correlated. Reward comparisons that control for initialization also show that aggregating across posteriors can invert the ranking that a reward implies within any single posterior. Lastly, on 45 enumerated posteriors with three training seeds each, exact and sampled gradients of the same trajectory objective reduce mean physical estimation variance by $12.63\%$ and $6.32\%$ relative to a tuned mixture, while averaging the trajectory score without rescaling its penalty yields only $0.43\%$. This indicates that objective scaling accounts for part of the disparity between sampled and exact training. The regained benefit is concentrated at four shots and flips sign at sixteen, so these exact, visible-bank diagnostics identify a concrete, reproducible failure mode in sampled training, the objective-scaling fix that addresses part of it, and the remaining gap.

Abstract:Reward-based learning can alter the very posterior distribution that scientific inference aims to estimate. GRPO-QM sidesteps this by learning only an exploration strategy for a stated quantum-tomography posterior: a group-relative policy chooses among reversible physical moves, and an exact Metropolis correction ensures the posterior remains stationary once the policy is fixed. We then examine what learning contributes beyond physical proposal mechanisms and prior knowledge. Reconstruction comparisons suggest that most of the gains over the tested flows come from those two components rather than from learning itself, and a closed-form counterexample explains why: a reward tied to accepted motion can increase even when a physical observable remains highly correlated. Reward comparisons that control for initialization also show that aggregating across posteriors can invert the ranking that a reward implies within any single posterior. Lastly, on 45 enumerated posteriors with three training seeds each, exact and sampled gradients of the same trajectory objective reduce mean physical estimation variance by $12.63\%$ and $6.32\%$ relative to a tuned mixture, while averaging the trajectory score without rescaling its penalty yields only $0.43\%$. This indicates that objective scaling accounts for part of the disparity between sampled and exact training. The regained benefit is concentrated at four shots and flips sign at sixteen, so these exact, visible-bank diagnostics identify a concrete, reproducible failure mode in sampled training, the objective-scaling fix that addresses part of it, and the remaining gap.

Machine Learning (cs.LG); Quantum Physics (quant-ph)

arXiv:2609.14711 [cs.LG]

(or arXiv:2609.14711v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2609.14711 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

GRPO-QM: Target Preserving Exploration for Quantum Tomography

Related coverage