论文标题
从特权的半生产MRI图像中学习多模式脑肿瘤分割的课程分解学习
Learning Multi-Modal Brain Tumor Segmentation from Privileged Semi-Paired MRI Images with Curriculum Disentanglement Learning
论文作者
论文摘要
由于在临床实践中获得多模式成对图像的困难，最近的研究提议使用未配对的图像训练脑肿瘤分割模型，并通过模态翻译捕获互补信息。但是，这些模型无法完全利用不同方式的互补信息。因此，在这项工作中，我们提出了一个新型的两步（内模态和模式间）课程分离学习框架，以有效利用特权的半成配图像，即仅在训练中可用于脑肿瘤分段的有限的配对图像。具体而言，在第一步中，我们建议使用增强的模式内风格一致性图像进行重建和分割。在第二步中，该模型共同执行重建，无监督/监督的翻译以及对未配对和配对模式图像的分割。提出了内容一致性损失和监督翻译损失，以利用此步骤中不同方式的互补信息。通过这两个步骤，我们的方法有效地提取了特定于模式的样式代码，描述了组织特征和图像对比度的衰减以及模态不变的内容代码，其中包含来自输入图像的解剖和功能信息。对三个脑肿瘤分割任务的实验表明，我们的模型比基于未配对的图像优于竞争分割模型。
Due to the difficulties of obtaining multimodal paired images in clinical practice, recent studies propose to train brain tumor segmentation models with unpaired images and capture complementary information through modality translation. However, these models cannot fully exploit the complementary information from different modalities. In this work, we thus present a novel two-step (intra-modality and inter-modality) curriculum disentanglement learning framework to effectively utilize privileged semi-paired images, i.e. limited paired images that are only available in training, for brain tumor segmentation. Specifically, in the first step, we propose to conduct reconstruction and segmentation with augmented intra-modality style-consistent images. In the second step, the model jointly performs reconstruction, unsupervised/supervised translation, and segmentation for both unpaired and paired inter-modality images. A content consistency loss and a supervised translation loss are proposed to leverage complementary information from different modalities in this step. Through these two steps, our method effectively extracts modality-specific style codes describing the attenuation of tissue features and image contrast, and modality-invariant content codes containing anatomical and functional information from the input images. Experiments on three brain tumor segmentation tasks show that our model outperforms competing segmentation models based on unpaired images.
