analiza aiqbench
This research introduces "trajectory balance," a new learning objective for Generative Flow Networks (GFlowNets) designed to solve credit assignment inefficiencies in long action sequences. By proving that this method enables exact sampling from target distributions, the study offers a significant improvement in convergence speed and robustness across diverse domains like graph generation. This advancement directly enhances the scalability of generative models for complex compositional tasks.
Trajectory balance: Improved credit assignment in GFlowNets
arXiv:2201.13259v4 Announce Type: replace Abstract: Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object. We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences. We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution. In experiments on four distinct domains, we empirically demonstrate the benefits of the trajectory balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces.
Abstract:Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object. We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences. We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution. In experiments on four distinct domains, we empirically demonstrate the benefits of the trajectory balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces.
Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2201.13259 [cs.LG]
(or arXiv:2201.13259v4 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2201.13259 Focus to learn more arXiv-issued DOI via DataCite
https://doi.org/10.52202/068431-0431 Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 252 cuvinte
Subiecte conexe
Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training
arXiv · 21 iul. 2026
Can the Environment Speak for Itself? $T^{2}$-GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents
arXiv · 9 iun. 2026
CDPR: Counterfactual Advantage-based Credit Assignment for Cost-Aware Sequential Medical Diagnosis
arXiv · 1 sept. 2026
Action abstractions for amortized sampling
arXiv · 3 sept. 2026