论文标题
通过过滤器内的重量共享通过重量分享的可区分通道稀疏搜索
Differentiable Channel Sparsity Search via Weight Sharing within Filters
论文作者
论文摘要
在本文中，我们建议用于卷积神经网络的可区分通道稀疏搜索（DCSS）。与传统的频道修剪算法不同，该算法要求用户手动为每个卷积层设置修剪比率，DCSS会自动搜索稀疏度的最佳组合。受到可区分架构搜索（飞镖）的启发，我们从持续放松中汲取了教训，并利用梯度信息来平衡计算成本和指标。由于直接应用飞镖方案会导致形状不匹配和过度记忆消耗，因此我们引入了一种新型技术，称为过滤器中的重量共享。该技术优雅地消除了形状不匹配的问题，而其他资源可忽略不计。我们不仅对图像分类进行了全面的实验，还进行了包括语义分割和图像超级分辨率在内的详细任务，以验证DCSS的有效性。与以前的网络修剪方法相比，DCSS取得了图像分类的最新结果。语义分割和图像超级分辨率的实验结果表明，特定于任务的搜索比转移纤细模型更好，这表明了DCS的广泛适用性和高效率。
In this paper, we propose the differentiable channel sparsity search (DCSS) for convolutional neural networks. Unlike traditional channel pruning algorithms which require users to manually set prune ratios for each convolutional layer, DCSS automatically searches the optimal combination of sparsities. Inspired by the differentiable architecture search (DARTS), we draw lessons from the continuous relaxation and leverage the gradient information to balance the computational cost and metrics. Since directly applying the scheme of DARTS causes shape mismatching and excessive memory consumption, we introduce a novel technique called weight sharing within filters. This technique elegantly eliminates the problem of shape mismatching with negligible additional resources. We conduct comprehensive experiments on not only image classification but also find-grained tasks including semantic segmentation and image super resolution to verify the effectiveness of DCSS. Compared with previous network pruning approaches, DCSS achieves state-of-the-art results for image classification. Experimental results of semantic segmentation and image super resolution indicate that task-specific search achieves better performance than transferring slim models, demonstrating the wide applicability and high efficiency of DCSS.
