论文标题
与政策代表团的强大层次结构计划
Robust Hierarchical Planning with Policy Delegation
论文作者
论文摘要
我们根据代表团原则提出了一种新颖的框架和算法，以用于等级规划。这个框架是马尔可夫的意图过程，具有一系列技能，每个技能都专门执行一个任务。技能意识到了他们的预期效果，并能够分析计划目标，以将计划委托给最佳的技能。该原则动态地创建了一个计划的层次结构，其中的每个技能计划都专门为子目标。所提出的计划方法功能按需执行---技能策略只有在需要时进行评估。计划仅在最高级别生成，然后在可用的最新状态信息时进行扩展和优化。高级计划保留了最初的计划意图和先前计算的技能，从而有效地减少了适应环境变化所需的计算。我们表明，这种计划方法在实验上对在解决方案的长度和计划时间方面的经典计划和强化学习技术都非常有竞争力。
We propose a novel framework and algorithm for hierarchical planning based on the principle of delegation. This framework, the Markov Intent Process, features a collection of skills which are each specialised to perform a single task well. Skills are aware of their intended effects and are able to analyse planning goals to delegate planning to the best-suited skill. This principle dynamically creates a hierarchy of plans, in which each skill plans for sub-goals for which it is specialised. The proposed planning method features on-demand execution---skill policies are only evaluated when needed. Plans are only generated at the highest level, then expanded and optimised when the latest state information is available. The high-level plan retains the initial planning intent and previously computed skills, effectively reducing the computation needed to adapt to environmental changes. We show this planning approach is experimentally very competitive to classic planning and reinforcement learning techniques on a variety of domains, both in terms of solution length and planning time.
