论文标题
在游泳者上进行加强学习工作
Making Reinforcement Learning Work on Swimmer
论文作者
论文摘要
游泳者环境是增强学习（RL）的标准基准。特别是，它通常用于比较或组合RL方法与直接策略搜索方法（例如遗传算法或进化策略）的论文中。这些论文中的许多论文都报告说，来自RL方法的游泳者表现不佳，直接策略搜索方法的性能更好。在这份技术报告中，我们表明，游泳者的RL方法的性能较低，这仅来自重要的超参数（折扣系数）的调整不足。此外，我们表明，通过将此超参数设置为正确的值，可以轻松解决该问题。最后，对于一组经常使用的RL算法，我们提供了一组成功的超参数，可通过稳定的Baselines3库及其RL动物园获得。
The SWIMMER environment is a standard benchmark in reinforcement learning (RL). In particular, it is often used in papers comparing or combining RL methods with direct policy search methods such as genetic algorithms or evolution strategies. A lot of these papers report poor performance on SWIMMER from RL methods and much better performance from direct policy search methods. In this technical report we show that the low performance of RL methods on SWIMMER simply comes from the inadequate tuning of an important hyper-parameter, the discount factor. Furthermore we show that, by setting this hyper-parameter to a correct value, the issue can be easily fixed. Finally, for a set of often used RL algorithms, we provide a set of successful hyper-parameters obtained with the Stable Baselines3 library and its RL Zoo.
