论文标题
基于神经切线内核的单层对抗数据综合
Single-level Adversarial Data Synthesis based on Neural Tangent Kernels
论文作者
论文摘要
摘要生成对抗网络（GAN）在数据综合方面取得了令人印象深刻的性能，并驱动了许多应用程序的开发。但是，由于其双重目标，甘恩很难训练，这导致了收敛，模式崩溃和梯度消失的问题。在本文中，我们提出了一种具有单层目标的新生成模型，称为“生成对抗NTK（GA-NTK）”。 GA-NTK保持对抗性学习的精神（这有助于产生合理的数据），同时避免了gan的训练困难。这是通过用神经切线内核（NTK-GP）建模歧视器作为高斯过程来完成的，其训练动力可以通过封闭形式的公式完全描述。我们分析了通过梯度下降训练的GA-NTK的收敛行为，并提供了一些足够的收敛条件。我们还进行了广泛的实验来研究GA-NTK的优势和局限性，并提出了一些使Ga-NTK更实用的技术。
Abstract Generative adversarial networks (GANs) have achieved impressive performance in data synthesis and have driven the development of many applications. However, GANs are known to be hard to train due to their bilevel objective, which leads to the problems of convergence, mode collapse, and gradient vanishing. In this paper, we propose a new generative model called the generative adversarial NTK (GA-NTK) that has a single-level objective. The GA-NTK keeps the spirit of adversarial learning (which helps generate plausible data) while avoiding the training difficulties of GANs. This is done by modeling the discriminator as a Gaussian process with a neural tangent kernel (NTK-GP) whose training dynamics can be completely described by a closed-form formula. We analyze the convergence behavior of GA-NTK trained by gradient descent and give some sufficient conditions for convergence. We also conduct extensive experiments to study the advantages and limitations of GA-NTK and propose some techniques that make GA-NTK more practical.
