aiqbench analysis
This research introduces a novel policy-based training framework for Generative Flow Networks (GFlowNets) that unifies flow balance constraints with expected reward optimization, addressing efficiency issues in backward policy design. By jointly solving forward and backward policy training strategies, the method offers theoretical guarantees and robust gradient estimation verified across simulated and real-world datasets. The work advances reinforcement learning perspectives specifically tailored for generating combinatorial objects with desired properties.
GFlowNet Training by Policy Gradients
arXiv:2408.05885v3 Announce Type: replace Abstract: Generative Flow Networks (GFlowNets) have been shown effective to generate combinatorial objects with desired properties. We here propose a new GFlowNet training framework, with policy-dependent rewards, that bridges keeping flow balance of GFlowNets to optimizing the expected accumulated reward in traditional Reinforcement-Learning (RL). This enables the derivation of new policy-based GFlowNet training methods, in contrast to existing ones resembling value-based RL. It is known that the design of backward policies in GFlowNet training affects efficiency. We further develop a coupled training strategy that jointly solves GFlowNet forward policy training and backward policy design. Performance analysis is provided with a theoretical guarantee of our policy-based GFlowNet training. Experiments on both simulated and real-world datasets verify that our policy-based strategies provide advanced RL perspectives for robust gradient estimation to improve GFlowNet performance.
Abstract:Generative Flow Networks (GFlowNets) have been shown effective to generate combinatorial objects with desired properties. We here propose a new GFlowNet training framework, with policy-dependent rewards, that bridges keeping flow balance of GFlowNets to optimizing the expected accumulated reward in traditional Reinforcement-Learning (RL). This enables the derivation of new policy-based GFlowNet training methods, in contrast to existing ones resembling value-based RL. It is known that the design of backward policies in GFlowNet training affects efficiency. We further develop a coupled training strategy that jointly solves GFlowNet forward policy training and backward policy design. Performance analysis is provided with a theoretical guarantee of our policy-based GFlowNet training. Experiments on both simulated and real-world datasets verify that our policy-based strategies provide advanced RL perspectives for robust gradient estimation to improve GFlowNet performance.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2408.05885 [cs.LG]
(or arXiv:2408.05885v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2408.05885 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 225 words
Related coverage
Proximal Policy Optimization for Amortized Discrete Sampling
arXiv · Jun 16, 2026
Information-Geometric Forward Policy Training in GFlowNets
arXiv · Aug 5, 2026
Mixture of GFlowNets
arXiv · Oct 14, 2026
Interpreting GFlowNets for Drug Discovery: What probes can and cannot show
arXiv · Aug 6, 2026