论文标题
通过任务关系建模的多代理政策转移
Multi-Agent Policy Transfer via Task Relationship Modeling
论文作者
论文摘要
团队适应新的合作任务是人类智能的标志，在学习代理商中尚未完全实现。以前关于多机构转移学习的工作可容纳不同大小的团队，在很大程度上依靠神经网络的概括能力来适应看不见的任务。我们认为，任务之间的关系为策略适应提供了关键信息。在本文中，我们尝试使用固定的培训方案来发现和利用任务之间的共同结构，以进行更有效的转移，并提议将基于效果的任务表示作为任务的共同空间。我们证明任务表示可以捕获任务之间的关系，并可以推广到看不见的任务。结果，提出的方法可以在培训一些源任务后帮助将学习的合作知识转移到新任务中。我们还发现，调整转移的政策有助于解决难以从头开始学习的任务。
Team adaptation to new cooperative tasks is a hallmark of human intelligence, which has yet to be fully realized in learning agents. Previous work on multi-agent transfer learning accommodate teams of different sizes, heavily relying on the generalization ability of neural networks for adapting to unseen tasks. We believe that the relationship among tasks provides the key information for policy adaptation. In this paper, we try to discover and exploit common structures among tasks for more efficient transfer, and propose to learn effect-based task representations as a common space of tasks, using an alternatively fixed training scheme. We demonstrate that the task representation can capture the relationship among tasks, and can generalize to unseen tasks. As a result, the proposed method can help transfer learned cooperation knowledge to new tasks after training on a few source tasks. We also find that fine-tuning the transferred policies help solve tasks that are hard to learn from scratch.
