论文标题
通过路径正规化改善生成流网络
Improving Generative Flow Networks with Path Regularization
论文作者
论文摘要
生成流动网络（Gflownets）是最近提出的学习随机策略的模型，该模型通过与给定奖励功能成正比的动作序列生成组成对象。 Gflownets的核心问题是改善其探索和概括。在这项工作中，我们提出了一种基于最佳运输理论的新型路径正则化方法，该方法对Gflownets的基础结构施加了先前的约束。先验旨在帮助Gflownets更好地发现目标分布的潜在结构或增强其在主动学习中探索环境的能力。该路径正规化控制Gflownets中的流量，以最大程度地提高两个正向策略之间的最佳传输距离或通过最小化最佳运输距离来改善概括，从而产生更多样化和新颖的候选者。此外，我们通过在特定情况下找到其封闭形式的解决方案和有意义的上限来得出正则化的有效实现，该界限可用作最小化正则化项的近似值。我们从经验上证明了路径正则化在各种任务上的优势，包括合成高网络环境建模，离散概率建模和生物学序列设计。
Generative Flow Networks (GFlowNets) are recently proposed models for learning stochastic policies that generate compositional objects by sequences of actions with the probability proportional to a given reward function. The central problem of GFlowNets is to improve their exploration and generalization. In this work, we propose a novel path regularization method based on optimal transport theory that places prior constraints on the underlying structure of the GFlowNets. The prior is designed to help the GFlowNets better discover the latent structure of the target distribution or enhance its ability to explore the environment in the context of active learning. The path regularization controls the flow in GFlowNets to generate more diverse and novel candidates via maximizing the optimal transport distances between two forward policies or to improve the generalization via minimizing the optimal transport distances. In addition, we derive an efficient implementation of the regularization by finding its closed form solutions in specific cases and a meaningful upper bound that can be used as an approximation to minimize the regularization term. We empirically demonstrate the advantage of our path regularization on a wide range of tasks, including synthetic hypergrid environment modeling, discrete probabilistic modeling, and biological sequence design.
