论文标题
人口游戏中独立学习者的进化动态
The Evolutionary Dynamics of Independent Learning Agents in Population Games
论文作者
论文摘要
在多代理设置下，了解强化学习的进化动态一直是一个开放的问题。虽然以前的作品主要集中在2播放器游戏上，但我们考虑了人口游戏，这对包括小型和匿名代理的大人群的战略性相互作用进行了建模。本文介绍了基于奖励信号推理的随机过程与独立学习者的动态之间的正式关系。使用主方程方法，我们提供了一个新颖的统一框架，用于通过单个部分微分方程（定理1）来表征种群动力学。通过涉及跨学习剂的案例研究，我们说明定理1使我们能够识别质量不同的进化动力学，分析稳态并深入了解人群的预期行为。此外，我们提出了广泛的实验结果，证实了定理1为各种学习方法和人群游戏所持有的。
Understanding the evolutionary dynamics of reinforcement learning under multi-agent settings has long remained an open problem. While previous works primarily focus on 2-player games, we consider population games, which model the strategic interactions of a large population comprising small and anonymous agents. This paper presents a formal relation between stochastic processes and the dynamics of independent learning agents who reason based on the reward signals. Using a master equation approach, we provide a novel unified framework for characterising population dynamics via a single partial differential equation (Theorem 1). Through a case study involving Cross learning agents, we illustrate that Theorem 1 allows us to identify qualitatively different evolutionary dynamics, to analyse steady states, and to gain insights into the expected behaviour of a population. In addition, we present extensive experimental results validating that Theorem 1 holds for a variety of learning methods and population games.
