论文标题
修剪 - 搜索：通过渠道修剪和结构重新聚集的有效的神经体系结构搜索
Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization
论文作者
论文摘要
神经体系结构搜索（NAS）和网络修剪是广泛研究的有效AI技术，但尚不完美。 NAS执行详尽的候选架构搜索，并产生了巨大的搜索成本。尽管（结构化的）修剪可以简单地缩小模型维度，但尚不清楚如何自动和最佳地决定每层稀疏性。在这项工作中，我们重新审视了图层宽度优化的问题，并提出了修剪 - 搜索（PAS），这是一种端到端的通道修剪方法，可以自动有效地搜索所需的子网络。具体而言，我们添加了深度二进制卷积，以直接通过梯度下降来学习修剪政策。通过结合结构重新聚体化和PA，我们成功地搜索了一个新的类似VGG和轻量级网络的家族，这可以使任意宽度相对于每个阶段而不是每个阶段的灵活性。实验结果表明，在Imagenet-1000分类任务上，我们提出的架构的表现优于先前的艺术$ 1.0 \％$ $ TOP-1的精度。此外，我们证明了对复杂任务（包括实例分割和图像翻译）的宽度搜索的有效性。代码和模型已发布。
Neural architecture search (NAS) and network pruning are widely studied efficient AI techniques, but not yet perfect. NAS performs exhaustive candidate architecture search, incurring tremendous search cost. Though (structured) pruning can simply shrink model dimension, it remains unclear how to decide the per-layer sparsity automatically and optimally. In this work, we revisit the problem of layer-width optimization and propose Pruning-as-Search (PaS), an end-to-end channel pruning method to search out desired sub-network automatically and efficiently. Specifically, we add a depth-wise binary convolution to learn pruning policies directly through gradient descent. By combining the structural reparameterization and PaS, we successfully searched out a new family of VGG-like and lightweight networks, which enable the flexibility of arbitrary width with respect to each layer instead of each stage. Experimental results show that our proposed architecture outperforms prior arts by around $1.0\%$ top-1 accuracy under similar inference speed on ImageNet-1000 classification task. Furthermore, we demonstrate the effectiveness of our width search on complex tasks including instance segmentation and image translation. Code and models are released.
