analiza aiqbench
This research addresses optimization instability in flow matching models by demonstrating that prediction-loss alignment ensures the integrability of gradient moments, even under uniform sampling conditions. By reconciling theoretical singularities with empirical success, the study provides a principled method to achieve more robust training across continuous and binary data settings. The corrected results confirm that aligned objectives remain trainable regardless of the specific sampler used, offering a clearer path for stable generative model development.
Prediction--Loss Alignment for Sampler--Robust Flow Matching Training
arXiv:2602.10420v4 Announce Type: replace Abstract: Recent work has popularized a practical recipe in diffusion and flow matching: predict the clean signal $x$, convert it to a velocity, and train through a velocity-space loss. The conversion contains a singular endpoint amplification and therefore appears prone to unstable optimization, yet recent systems obtain strong empirical results with this recipe. We investigate this tension through the integrability of the pre-optimizer stochastic-gradient second moment. Under stated initialization conditions, the moment diverges under Uniform sampling; boundary-suppressing sampling can restore integrability under an additional upper-growth condition. We then show that prediction--loss alignment eliminates this conversion-induced source of non-integrability. Under a uniform moment bound, alignment yields a finite second moment for every timestep density, including Uniform sampling. Controlled experiments across continuous and binary settings reproduce the predicted sampler-dependent instability and show that aligned objectives remain trainable across the tested samplers. These results reconcile pointwise amplification with sampler-dependent empirical success and support alignment as a principled route to more robust flow-matching training.
Abstract:Recent work has popularized a practical recipe in diffusion and flow matching: predict the clean signal $x$, convert it to a velocity, and train through a velocity-space loss. The conversion contains a singular endpoint amplification and therefore appears prone to unstable optimization, yet recent systems obtain strong empirical results with this recipe. We investigate this tension through the integrability of the pre-optimizer stochastic-gradient second moment. Under stated initialization conditions, the moment diverges under Uniform sampling; boundary-suppressing sampling can restore integrability under an additional upper-growth condition. We then show that prediction--loss alignment eliminates this conversion-induced source of non-integrability. Under a uniform moment bound, alignment yields a finite second moment for every timestep density, including Uniform sampling. Controlled experiments across continuous and binary settings reproduce the predicted sampler-dependent instability and show that aligned objectives remain trainable across the tested samplers. These results reconcile pointwise amplification with sampler-dependent empirical success and support alignment as a principled route to more robust flow-matching training.
24 pages, 9 tables, 10 figures. This version corrects errors in the experimental evaluation and revises the affected results and conclusions. It supersedes earlier versions; readers should refer to the corrected results presented here
Machine Learning (cs.LG); Information Theory (cs.IT); Image and Video Processing (eess.IV); Signal Processing (eess.SP)
arXiv:2602.10420 [cs.LG]
(or arXiv:2602.10420v4 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 215 cuvinte
Subiecte conexe
News
Quantum Computing Report · 25 iul. 2026
The current bottleneck is political will, not research
AI Alignment Forum · 11 iul. 2026
Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
arXiv · 29 sept. 2026
Co-Evolving Paths and Flows via Path-Flow Alignment
arXiv · 7 oct. 2026