论文标题
基于LS-DC损失的统一SVM算法
Unified SVM Algorithm Based on LS-DC Loss
论文作者
论文摘要
在过去的二十年中，支持向量机（SVM）已成为一种流行的监督机器学习模型，并且基于SVM模型的不同KKT条件，用于分类/回归的不同KKT条件，以不同的损失（包括凸损失或非convex损失）设计了许多不同的算法。在本文中，我们提出了一种算法，该算法可以在\ emph {unified}方案中训练不同的SVM模型。 First, we introduce a definition of the \emph{LS-DC} (\textbf{l}east \textbf{s}quares type of \textbf{d}ifference of \textbf{c}onvex) loss and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss.然后，基于DCA（凸算法差异），我们提出了一种称为\ emph {unisvm}的统一算法，该算法可以用任何凸或convex ls-dc损耗来求解SVM模型，其中仅计算出一个向量，尤其是通过特定选择的丢失进行计算。特别是，对于训练具有非凸损耗的强大SVM模型，UNISVM比所有现有算法都具有主导优势，因为它在迭代中具有封闭形式的解决方案，而现有算法始终需要解决L1SVM/L2SVM。此外，通过核基质的低级别近似，Unisvm可以以效率解决大规模的非线性问题。为了验证所提出算法的功效和可行性，我们对一些小的人造问题进行了许多实验，以及一些具有/没有离群值的大型基准任务，用于分类和回归，以与先进的算法进行比较。实验结果表明，UNISVM可以在更少的训练时间内实现可比的性能。 UNISVM的最大优点是其MATLAB中的核心代码小于10行；因此，用户或研究人员很容易掌握它。
Over the past two decades, support vector machine (SVM) has become a popular supervised machine learning model, and plenty of distinct algorithms are designed separately based on different KKT conditions of the SVM model for classification/regression with different losses, including the convex loss or nonconvex loss. In this paper, we propose an algorithm that can train different SVM models in a \emph{unified} scheme. First, we introduce a definition of the \emph{LS-DC} (\textbf{l}east \textbf{s}quares type of \textbf{d}ifference of \textbf{c}onvex) loss and show that the most commonly used losses in the SVM community are LS-DC loss or can be approximated by LS-DC loss. Based on DCA (difference of convex algorithm), we then propose a unified algorithm, called \emph{UniSVM}, which can solve the SVM model with any convex or nonconvex LS-DC loss, in which only a vector is computed, especially by the specifically chosen loss. Particularly, for training robust SVM models with nonconvex losses, UniSVM has a dominant advantage over all existing algorithms because it has a closed-form solution per iteration, while the existing algorithms always need to solve an L1SVM/L2SVM per iteration. Furthermore, by the low-rank approximation of the kernel matrix, UniSVM can solve the large-scale nonlinear problems with efficiency. To verify the efficacy and feasibility of the proposed algorithm, we perform many experiments on some small artificial problems and some large benchmark tasks with/without outliers for classification and regression for comparison with state-of-the-art algorithms. The experimental results demonstrate that UniSVM can achieve comparable performance in less training time. The foremost advantage of UniSVM is that its core code in Matlab is less than 10 lines; hence, it can be easily grasped by users or researchers.
