论文标题
FBNETV2：可区分的神经体系结构搜索空间和通道维度
FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions
论文作者
论文摘要
可区分的神经体系结构搜索（DNA）在设计最先进的，有效的神经网络方面取得了巨大的成功。但是，与其他搜索方法相比，基于飞镖的DNA的搜索空间很小，因为所有候选网络层都必须在内存中明确实例化。为了解决此瓶颈，我们提出了一个内存和计算有效的DNA变体：DMASKINGNAS。该算法将搜索空间扩展到$ 10^{14} \ times $上，而不是常规DNA，支持对空间和通道尺寸的搜索，这些搜索本来是昂贵的：输入分辨率和过滤器数量。我们提出了一种用于功能地图重复使用的掩蔽机制，以便随着搜索空间的扩展，内存和计算成本几乎保持恒定。此外，我们采用有效的形状传播来最大程度地提高每参数的准确性。与所有以前的架构相比，搜索的FBNETV2S产生最先进的性能。 Dmaskingnas的搜索成本最高为421 $ \ times $ $ \ times $，其精度高0.9％，拖失lop比Mobilenetv3-small少15％；并且准确性相似，但比有效B0少20％。此外，我们的FBNETV2的准确性优于MobilenetV3，具有同等的模型大小。 FBNETV2型号在https://github.com/facebookresearch/mobile-vision上开源。
Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space is small when compared to other search methods', since all candidate network layers must be explicitly instantiated in memory. To address this bottleneck, we propose a memory and computationally efficient DNAS variant: DMaskingNAS. This algorithm expands the search space by up to $10^{14}\times$ over conventional DNAS, supporting searches over spatial and channel dimensions that are otherwise prohibitively expensive: input resolution and number of filters. We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands. Furthermore, we employ effective shape propagation to maximize per-FLOP or per-parameter accuracy. The searched FBNetV2s yield state-of-the-art performance when compared with all previous architectures. With up to 421$\times$ less search cost, DMaskingNAS finds models with 0.9% higher accuracy, 15% fewer FLOPs than MobileNetV3-Small; and with similar accuracy but 20% fewer FLOPs than Efficient-B0. Furthermore, our FBNetV2 outperforms MobileNetV3 by 2.6% in accuracy, with equivalent model size. FBNetV2 models are open-sourced at https://github.com/facebookresearch/mobile-vision.
