论文标题
通过基于上下文的深度强化学习来检测和适应危机模式
Detecting and adapting to crisis pattern with context based Deep Reinforcement Learning
论文作者
论文摘要
深度强化学习（DRL）在复杂的任务（例如GON和自动驾驶）等复杂任务中达到了超级人为水平。但是，DRL是否可以在财务问题上的应用中达到人类水平，尤其是在检测模式危机和因此撤销投资方面仍然是一个悬而未决的问题。在本文中，我们提出了一个创新的DRL框架，该框架分别由两个子网络组成，分别采用了过去的表演和标准偏差以及其他上下文功能的投资组合策略。第二个子网络起着重要的作用，因为它捕获了具有常见财务指标的依赖性，例如规避风险，经济惊喜指数和资产之间的相关性，这些资产允许考虑基于上下文的信息。我们比较不同的网络体系结构，要么使用卷积层来降低网络的复杂性，要么使用LSTM块来捕获时间依赖性以及以前的分配在建模中是否重要。我们还使用对抗性训练来使最终模型更加强大。测试集中的结果表明，这种方法大大表现了Markowitz等传统投资组合优化方法，并且能够检测和预测像当前Covid一样的危机。
Deep reinforcement learning (DRL) has reached super human levels in complex tasks like game solving (Go and autonomous driving). However, it remains an open question whether DRL can reach human level in applications to financial problems and in particular in detecting pattern crisis and consequently dis-investing. In this paper, we present an innovative DRL framework consisting in two sub-networks fed respectively with portfolio strategies past performances and standard deviations as well as additional contextual features. The second sub network plays an important role as it captures dependencies with common financial indicators features like risk aversion, economic surprise index and correlations between assets that allows taking into account context based information. We compare different network architectures either using layers of convolutions to reduce network's complexity or LSTM block to capture time dependency and whether previous allocations is important in the modeling. We also use adversarial training to make the final model more robust. Results on test set show this approach substantially over-performs traditional portfolio optimization methods like Markowitz and is able to detect and anticipate crisis like the current Covid one.
