论文标题
3D指导弱监督语义细分
3D Guided Weakly Supervised Semantic Segmentation
论文作者
论文摘要
对于完全监督语义细分，必须使用像素清洁的注释，这是费力且昂贵的。在本文中，我们通过将稀疏边界框标签与可用的3D信息结合在一起，提出了一个弱监督的2D语义分割模型，使用高级传感器可以更容易获得。我们手动标记了带有边界框的2d-3d语义（2d-3d-S）数据集的子集，并介绍我们的2d-3d推理模块以生成准确的像素段建议蒙版。在3D信息的指导下，我们首先生成一个对象的点云，并计算每个点的对象概率分数。然后，我们将点云的物体概率投射回2D图像，然后进行改进步骤，以获取段建议，该建议被视为伪标记，以训练语义分割网络。我们的方法以递归方式起作用，逐渐完善上述节段建议。 2d-3d-S数据集上的广泛实验结果表明，当仅在一小部分训练图像上提供边界框标签时，提出的方法可以生成准确的段建议。绩效比较与最新的最新方法进一步说明了我们方法的有效性。
Pixel-wise clean annotation is necessary for fully-supervised semantic segmentation, which is laborious and expensive to obtain. In this paper, we propose a weakly supervised 2D semantic segmentation model by incorporating sparse bounding box labels with available 3D information, which is much easier to obtain with advanced sensors. We manually labeled a subset of the 2D-3D Semantics(2D-3D-S) dataset with bounding boxes, and introduce our 2D-3D inference module to generate accurate pixel-wise segment proposal masks. Guided by 3D information, we first generate a point cloud of objects and calculate objectness probability score for each point. Then we project the point cloud with objectness probabilities back to 2D images followed by a refinement step to obtain segment proposals, which are treated as pseudo labels to train a semantic segmentation network. Our method works in a recursive manner to gradually refine the above-mentioned segment proposals. Extensive experimental results on the 2D-3D-S dataset show that the proposed method can generate accurate segment proposals when bounding box labels are available on only a small subset of training images. Performance comparison with recent state-of-the-art methods further illustrates the effectiveness of our method.
