论文标题
结构化的分层对话政策与图神经网络
Structured Hierarchical Dialogue Policy with Graph Neural Networks
论文作者
论文摘要
对综合任务的对话政策培训，例如多个地方的餐厅预订，是一个重要且具有挑战性的问题。最近，层次深度强化学习（HDRL）方法在复合任务中取得了良好的性能。但是，在香草hdrl中，顶级和低级策略均由多层感知器（MLP）表示，这些策略将来自环境的所有观测值的串联作为预测动作的输入。因此，传统的HDRL方法通常患有低采样效率和可传递性差。在本文中，我们通过利用图神经网络（GNN）的灵活性来解决这些问题。提出了一种新颖的共网，以建模层次剂的结构。 Comnet的性能经过层状基准的合成任务进行了测试。实验表明，Comnet的表现优于Vanilla HDRL系统，其性能接近上限。它不仅可以达到样本效率，而且在保持对其他复合任务的转移性的同时，对噪声也更加强大。
Dialogue policy training for composite tasks, such as restaurant reservation in multiple places, is a practically important and challenging problem. Recently, hierarchical deep reinforcement learning (HDRL) methods have achieved good performance in composite tasks. However, in vanilla HDRL, both top-level and low-level policies are all represented by multi-layer perceptrons (MLPs) which take the concatenation of all observations from the environment as the input for predicting actions. Thus, traditional HDRL approach often suffers from low sampling efficiency and poor transferability. In this paper, we address these problems by utilizing the flexibility of graph neural networks (GNNs). A novel ComNet is proposed to model the structure of a hierarchical agent. The performance of ComNet is tested on composited tasks of the PyDial benchmark. Experiments show that ComNet outperforms vanilla HDRL systems with performance close to the upper bound. It not only achieves sample efficiency but also is more robust to noise while maintaining the transferability to other composite tasks.
