论文标题
深度学习与时空光谱聚类之间的迭代知识交流，以进行视频中的无监督分段
Iterative Knowledge Exchange Between Deep Learning and Space-Time Spectral Clustering for Unsupervised Segmentation in Videos
论文作者
论文摘要
我们为视频中的无监督对象分割提出了一个双重系统，该系统将两个模块与互补属性汇集在一起：一个时空图，可在视频中发现对象和一个学习强大对象功能的深层网络。该系统使用迭代知识交换政策。图表上的新型光谱时空聚类过程会产生无监督的分割口罩，以伪标记传递给网络。网络学会了单帧细分图表中发现的内容，并将其传递回图形强图像级特征，从而在下一个迭代中改善了其节点级特征。知识被交换为几个周期，直到收敛为止。该图每个视频像素具有一个节点，但是对象发现很快。它使用一种新型的功率迭代算法计算主要时空群集作为特殊特征 - 运动矩阵的主要特征向量，而无需实际计算矩阵。彻底的实验分析验证了我们的理论主张，并证明了周期性知识交流的有效性。我们还在监督方案上进行实验，并结合了通过人类监督预识别的特征。在四个具有挑战性的数据集上，我们在无监督和监督的方案上达到了最新的水平：戴维斯，塞格特拉克，YouTube-Objects和Davsod。
We propose a dual system for unsupervised object segmentation in video, which brings together two modules with complementary properties: a space-time graph that discovers objects in videos and a deep network that learns powerful object features. The system uses an iterative knowledge exchange policy. A novel spectral space-time clustering process on the graph produces unsupervised segmentation masks passed to the network as pseudo-labels. The net learns to segment in single frames what the graph discovers in video and passes back to the graph strong image-level features that improve its node-level features in the next iteration. Knowledge is exchanged for several cycles until convergence. The graph has one node per each video pixel, but the object discovery is fast. It uses a novel power iteration algorithm computing the main space-time cluster as the principal eigenvector of a special Feature-Motion matrix without actually computing the matrix. The thorough experimental analysis validates our theoretical claims and proves the effectiveness of the cyclical knowledge exchange. We also perform experiments on the supervised scenario, incorporating features pretrained with human supervision. We achieve state-of-the-art level on unsupervised and supervised scenarios on four challenging datasets: DAVIS, SegTrack, YouTube-Objects, and DAVSOD.
