论文标题
WPNA：神经体系结构搜索通过共同使用重量共享和预测指标
WPNAS: Neural Architecture Search by jointly using Weight Sharing and Predictor
论文作者
论文摘要
基于权重共享和基于预测指标的方法是快速神经体系结构搜索方法的两种主要类型。在本文中，我们建议在统一框架中共同使用重量共享和预测指标。首先，我们以体重分配的方式构造超级网，并从超级网中概率地采样体系结构。为了提高体系结构评估的正确性，除了使用遗传权重的直接评估外，我们还进一步应用了一些预测指标来评估体系结构。对体系结构的最终评估是直接评估的组合，预测指标和架构成本的预测。我们将评估视为奖励，并采用自我批评政策梯度方法来更新体系结构概率。为了进一步降低体重共享的副作用，我们通过引入另一个HyperNET提出了一种弱的重量共享方法。我们在Nats Bench，Darts和Mobilenet搜索空间的数据集上进行包括CIFAR-10，CIFAR-100和Imagenet在内的数据集进行实验。所提出的WPNA方法在这些数据集上实现了最先进的性能。
Weight sharing based and predictor based methods are two major types of fast neural architecture search methods. In this paper, we propose to jointly use weight sharing and predictor in a unified framework. First, we construct a SuperNet in a weight-sharing way and probabilisticly sample architectures from the SuperNet. To increase the correctness of the evaluation of architectures, besides direct evaluation using the inherited weights, we further apply a few-shot predictor to assess the architecture on the other hand. The final evaluation of the architecture is the combination of direct evaluation, the prediction from the predictor and the cost of the architecture. We regard the evaluation as a reward and apply a self-critical policy gradient approach to update the architecture probabilities. To further reduce the side effects of weight sharing, we propose a weakly weight sharing method by introducing another HyperNet. We conduct experiments on datasets including CIFAR-10, CIFAR-100 and ImageNet under NATS-Bench, DARTS and MobileNet search space. The proposed WPNAS method achieves state-of-the-art performance on these datasets.
