论文标题
稀疏高斯过程的直接损失最小化算法
Direct loss minimization algorithms for sparse Gaussian processes
论文作者
论文摘要
本文对直接损失最小化（DLM）进行了彻底的研究，该研究优化了稀疏高斯过程中的预测损失的后部。对于共轭案例，我们考虑对数损坏和DLM的DLM，用于正方形损耗，在这两种情况下显示出显着的性能提高。 DLM在非偶联的情况下的应用更为复杂，因为日志损坏DLM物镜中的期望对数通常很棘手，简单的采样会导致梯度的偏差估计值。该论文为解决这个问题做出了两项技术贡献。首先，提出了一种使用产品采样的新方法，该方法对目标函数的梯度（UPS）进行了无偏估计。其次，对有偏见的蒙特卡洛估计值（BMC）的理论分析表明，尽管有偏置梯度，但随机梯度下降会收敛。实验证明了DLM的经验成功。对采样方法的比较表明，尽管UPS可能更有效率，但BMC在收敛时间和计算效率方面提供了更好的权衡。
The paper provides a thorough investigation of Direct loss minimization (DLM), which optimizes the posterior to minimize predictive loss, in sparse Gaussian processes. For the conjugate case, we consider DLM for log-loss and DLM for square loss showing a significant performance improvement in both cases. The application of DLM in non-conjugate cases is more complex because the logarithm of expectation in the log-loss DLM objective is often intractable and simple sampling leads to biased estimates of gradients. The paper makes two technical contributions to address this. First, a new method using product sampling is proposed, which gives unbiased estimates of gradients (uPS) for the objective function. Second, a theoretical analysis of biased Monte Carlo estimates (bMC) shows that stochastic gradient descent converges despite the biased gradients. Experiments demonstrate empirical success of DLM. A comparison of the sampling methods shows that, while uPS is potentially more sample-efficient, bMC provides a better tradeoff in terms of convergence time and computational efficiency.
