论文标题
在代表空间中进行样品有效探索的新颖搜索
Novelty Search in Representational Space for Sample Efficient Exploration
论文作者
论文摘要
我们提出了一种有效探索的新方法，该方法利用了通过基于模型和无模型的目标组合来了解环境的低维编码。我们的方法使用基于低维代表空间中最近邻居的距离来衡量新颖性的固有奖励。然后，我们利用这些内在的奖励来进行样本效率的探索，并在代表空间中的计划例程，用于稀疏奖励的硬探索任务。我们方法的一个关键要素是使用信息理论原理以某种方式塑造我们的表示形式，以便我们的新颖性奖励超越像素的相似性。我们在许多迷宫任务上测试了我们的方法，以及一个控制问题，并表明与强质基线相比，我们的探索方法更有效期。
We present a new approach for efficient exploration which leverages a low-dimensional encoding of the environment learned with a combination of model-based and model-free objectives. Our approach uses intrinsic rewards that are based on the distance of nearest neighbors in the low dimensional representational space to gauge novelty. We then leverage these intrinsic rewards for sample-efficient exploration with planning routines in representational space for hard exploration tasks with sparse rewards. One key element of our approach is the use of information theoretic principles to shape our representations in a way so that our novelty reward goes beyond pixel similarity. We test our approach on a number of maze tasks, as well as a control problem and show that our exploration approach is more sample-efficient compared to strong baselines.
