aiqbench analysis
This research applies Proximal Policy Optimization (PPO) within the Generative Flow Network framework, demonstrating superior convergence speed and data efficiency over standard training objectives for discrete sampling tasks. By bridging theoretical connections between GFlowNets and entropy-regularized reinforcement learning, the study offers a practical advancement in generating complex structures like molecular graphs from synthetic energy benchmarks.
Proximal Policy Optimization for Amortized Discrete Sampling
arXiv:2606.15793v1 Announce Type: new Abstract: This paper explores policy gradient algorithms for training stochastic policies to sample from structured discrete probability distributions under the Generative Flow Network (GFlowNet) framework. Building on extensive theoretical connections between GFlowNets and entropy-regularized reinforcement learning, we derive equivalents of standard policy gradient algorithms for training GFlowNets, as well as experimentally explore their various methodological aspects, including baseline training and advantage estimation. Most importantly, our work is the first to derive and successfully apply proximal policy optimization to GFlowNets, showing its improved convergence speed and data efficiency compared to standard GFlowNet training objectives on benchmarks ranging from synthetic energies to molecular graph generation.
Abstract:This paper explores policy gradient algorithms for training stochastic policies to sample from structured discrete probability distributions under the Generative Flow Network (GFlowNet) framework. Building on extensive theoretical connections between GFlowNets and entropy-regularized reinforcement learning, we derive equivalents of standard policy gradient algorithms for training GFlowNets, as well as experimentally explore their various methodological aspects, including baseline training and advantage estimation. Most importantly, our work is the first to derive and successfully apply proximal policy optimization to GFlowNets, showing its improved convergence speed and data efficiency compared to standard GFlowNet training objectives on benchmarks ranging from synthetic energies to molecular graph generation.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
arXiv:2606.15793 [cs.LG]
(or arXiv:2606.15793v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.15793 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 202 words
Related coverage
Scaling RL with verl on AMD Instinct MI355X: Async Walkthrough and Sync Benchmark
AMD · Aug 18, 2026
Learn for Variation: Efficient AAV Trajectory Learning through a Differentiable Wireless World Model
arXiv · Aug 21, 2026
Constrained Command-Conditioned Reinforcement Learning with Bandit Strategy Selection in Real-Time Strategy Games
arXiv · Oct 10, 2026
Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNets
arXiv · Sep 1, 2026