论文标题
数据驱动的随机最佳控制使用线性转移操作员
Data-Driven Stochastic Optimal Control using Linear Transfer Operators
论文作者
论文摘要
我们提供了一个数据驱动的框架，以最佳控制连续的随机动力学系统。提出的框架依赖于涉及线性perron-frobenius（P-F）和Koopman运算符的线性操作者理论。我们的第一个涉及P-F操作员的结果为密度双重空间中最佳控制问题提供了凸公式。随机最佳控制问题的这种凸公式导致无限维凸面程序。凸面程序的有限维近似是使用P-F运算符的数据驱动近似获得的。我们的第二个结果证明了对P-F运算符双重的Koopman操作员在随机最佳控制设计中的使用。我们证明汉密尔顿雅各比·贝尔曼（HJB）方程可以使用库普曼操作员表示。我们根据流行策略迭代算法的界线提供了一个迭代过程，该算法基于用于求解HJB方程的Koopman操作员的数据驱动近似。这两个配方，即使用HJB方程的涉及P-F运算符和基于Koopman的公式的凸公式，可以将双重性视为双重性，而二元性由于P-F和Koopman运算符的双重性质而遵循。最后，我们提出了几个数值示例，以证明开发框架的功效。
We provide a data-driven framework for optimal control of a continuous-time stochastic dynamical system. The proposed framework relies on the linear operator theory involving linear Perron-Frobenius (P-F) and Koopman operators. Our first results involving the P-F operator provide a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the P-F operator. Our second results demonstrate the use of the Koopman operator, which is dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving P-F operator and Koopman based formulation using HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of P-F and Koopman operators. Finally, we present several numerical examples to demonstrate the efficacy of the developed framework.
