论文标题
在深度学习世界中解散小样本量问题
Unravelling Small Sample Size Problems in the Deep Learning World
论文作者
论文摘要
深度学习方法的增长和成功可以归因于两个主要因素：硬件资源的可用性和大量培训样本的可用性。对于大型培训数据库的问题，深度学习模型已经达到了最高级的表现。但是，有很多\ textIt {小样本量或$ s^3 $}问题，这些问题是不可行的。已经观察到，深度学习模型在$ s^3 $问题上并不能很好地概括，并且需要专门的解决方案。在本文中，我们首先介绍了针对小样本量问题的深度学习算法的评论，其中根据其操作的空间（即输入空间，模型空间和特征空间）隔离算法。其次，我们提出了动态的注意池方法，该方法的重点是从特征图的最歧视性子部分中提取全局信息。在相对较小的公开数据集（例如SVHN，C10，C100和Tinyimagenet）上，通过最先进的重新连接模型分析了提出的动态注意力集合的性能。
The growth and success of deep learning approaches can be attributed to two major factors: availability of hardware resources and availability of large number of training samples. For problems with large training databases, deep learning models have achieved superlative performances. However, there are a lot of \textit{small sample size or $S^3$} problems for which it is not feasible to collect large training databases. It has been observed that deep learning models do not generalize well on $S^3$ problems and specialized solutions are required. In this paper, we first present a review of deep learning algorithms for small sample size problems in which the algorithms are segregated according to the space in which they operate, i.e. input space, model space, and feature space. Secondly, we present Dynamic Attention Pooling approach which focuses on extracting global information from the most discriminative sub-part of the feature map. The performance of the proposed dynamic attention pooling is analyzed with state-of-the-art ResNet model on relatively small publicly available datasets such as SVHN, C10, C100, and TinyImageNet.
