论文标题
从替代损失分类减少的有效政策学习
Efficient Policy Learning from Surrogate-Loss Classification Reductions
论文作者
论文摘要
从观察数据中学习政策学习的最新工作强调了有效的政策评估的重要性，并提出了减少加权（成本敏感）分类的重要性。但是，有效的政策评估无需对策略参数的有效估计。我们考虑通过任何分数功能，直接，逆向加权或双重稳定性的任何分数功能，加权替代损失分类减少政策学习给出的估计问题。我们表明，在正确的规范假设下，加权分类公式无需对策略参数有效。我们与实际（可能是加权的）二进制分类形成对比，其中正确的规范意味着参数模型，而对于策略学习，它仅意味着一个半参数模型。鉴于此，我们改为提出了一种基于一般矩方法的估算方法，这对于策略参数有效。我们根据使用神经网络解决时刻问题的最新发展提出了一种特定的方法，并以经验证明了该方法的效率和后悔。
Recent work on policy learning from observational data has highlighted the importance of efficient policy evaluation and has proposed reductions to weighted (cost-sensitive) classification. But, efficient policy evaluation need not yield efficient estimation of policy parameters. We consider the estimation problem given by a weighted surrogate-loss classification reduction of policy learning with any score function, either direct, inverse-propensity weighted, or doubly robust. We show that, under a correct specification assumption, the weighted classification formulation need not be efficient for policy parameters. We draw a contrast to actual (possibly weighted) binary classification, where correct specification implies a parametric model, while for policy learning it only implies a semiparametric model. In light of this, we instead propose an estimation approach based on generalized method of moments, which is efficient for the policy parameters. We propose a particular method based on recent developments on solving moment problems using neural networks and demonstrate the efficiency and regret benefits of this method empirically.
