aiqbench analysis
This research introduces a novel reward-guided fine-tuning technique for one-step generative models using Wasserstein Gradient Flow, enabling stable distributional updates without requiring differentiable rewards. By addressing challenges like mode collapse and reward hacking through optimal transport theory, the method improves alignment on complex benchmarks such as ImageNet while maintaining computational efficiency.
Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow
arXiv:2608.29647v1 Announce Type: new Abstract: To mitigate the time complexity of generative models, one-step generative models have recently emerged through direct mapping from noise to data in a single forward pass. However, the reward-guided fine-tuning method of one-step generative models remains largely unexplored. To address this, we consider one-step generators from an optimal transport view, investigating Wasserstein Gradient Flow (WGF) for modeling smooth and controlled distributional evolution in probability space. We then propose a novel reward-guided fine-tuning of a one-step generative model via WGF. We derive a practical training method that requires no reward gradients, thereby handling both non-differentiable and differentiable rewards. Moreover, our method provides smooth and stable reward-guided distributional updates while mitigating reward hacking and mode collapse. Experiments on 2D synthetic data, CIFAR-10, and ImageNet 256$\times$256 with diverse rewards, including JPEG (in)compressibility, class probability, Black-and-White and CLIP alignment, show that our method achieves better reward alignment compared to baselines.
Abstract:To mitigate the time complexity of generative models, one-step generative models have recently emerged through direct mapping from noise to data in a single forward pass. However, the reward-guided fine-tuning method of one-step generative models remains largely unexplored. To address this, we consider one-step generators from an optimal transport view, investigating Wasserstein Gradient Flow (WGF) for modeling smooth and controlled distributional evolution in probability space. We then propose a novel reward-guided fine-tuning of a one-step generative model via WGF. We derive a practical training method that requires no reward gradients, thereby handling both non-differentiable and differentiable rewards. Moreover, our method provides smooth and stable reward-guided distributional updates while mitigating reward hacking and mode collapse. Experiments on 2D synthetic data, CIFAR-10, and ImageNet 256$\times$256 with diverse rewards, including JPEG (in)compressibility, class probability, Black-and-White and CLIP alignment, show that our method achieves better reward alignment compared to baselines.
Machine Learning (cs.LG)
arXiv:2608.29647 [cs.LG]
(or arXiv:2608.29647v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.29647 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 240 words
Related coverage
The art and science of hyperparameter optimization on Amazon Nova Forge
AWS · Jun 2, 2026
Presentation: Engineering AI for Creativity and Curiosity on Mobile
InfoQ · Jul 21, 2026
Granite 4.2 LLMs: How They're Built
Hugging Face · Aug 25, 2026
The generative AI customization spectrum: From prompt engineering to custom models on AWS
AWS · Sep 14, 2026