论文标题
对单个时间尺度演员评论家的少量增益分析
A Small Gain Analysis of Single Timescale Actor Critic
论文作者
论文摘要
我们考虑了一个版本的Actor-Critic版本，该版本使用比例的步骤尺寸，并且只有一个评论家更新每个Actor步骤中的固定发行版本的单个样本。我们使用小增生定理对此方法进行了分析。 Specifically, we prove that this method can be used to find a stationary point, and that the resulting sample complexity improves the state of the art for actor-critic methods to $O \left(μ^{-2} ε^{-2} \right)$ to find an $ε$-approximate stationary point where $μ$ is the condition number associated with the critic.
We consider a version of actor-critic which uses proportional step-sizes and only one critic update with a single sample from the stationary distribution per actor step. We provide an analysis of this method using the small-gain theorem. Specifically, we prove that this method can be used to find a stationary point, and that the resulting sample complexity improves the state of the art for actor-critic methods to $O \left(μ^{-2} ε^{-2} \right)$ to find an $ε$-approximate stationary point where $μ$ is the condition number associated with the critic.
