论文标题
使用暹罗神经网络进行临床自然语言处理的少量学习
Few-Shot Learning for Clinical Natural Language Processing Using Siamese Neural Networks
论文作者
论文摘要
临床自然语言处理（NLP）已成为医疗保健领域的一项新兴技术，该技术利用电子健康记录（EHR）中大量的自由文本数据来改善患者护理，支持临床决策并促进临床和转化科学研究。最近，深度学习在许多临床NLP任务中都取得了最先进的表现。但是，培训深度学习模型通常需要大量注释的数据集，这些数据集通常不公开，并且可能耗时以在临床领域建立。在临床NLP中，使用较小的注释数据集是典型的，因此，确保深度学习模型表现良好对于在现实世界应用中使用的模型至关重要。一种广泛采用的方法是微调现有的预训练的语言模型（PLM），但是当培训数据集仅包含少数注释的样本时，这些尝试不足。最近已经研究了很少的学习（FSL）来解决这个问题。暹罗神经网络（SNN）已被广泛用作计算机视觉中的FSL方法，但在NLP中尚未得到很好的研究。此外，关于其在临床领域中应用的文献很少。在本文中，我们提出了两种基于SNN的FSL方法，用于临床NLP，包括带有二阶嵌入（SOE-SNN）的预训练的SNN（PT-SNN）和SNN。我们评估了针对两项临床任务的拟议方法，即临床文本分类和临床命名实体识别。我们测试了三个几次弹药设置，包括4次射击，8局和16杆学习。使用三个PLM（包括Bert，Biobert和Bioclinicalbert）对这两个临床NLP任务进行了测试。实验结果验证了两个NLP任务中提出的基于SNN的FSL方法的有效性。
Clinical Natural Language Processing (NLP) has become an emerging technology in healthcare that leverages a large amount of free-text data in electronic health records (EHRs) to improve patient care, support clinical decisions, and facilitate clinical and translational science research. Recently, deep learning has achieved state-of-the-art performance in many clinical NLP tasks. However, training deep learning models usually requires large annotated datasets, which are normally not publicly available and can be time-consuming to build in clinical domains. Working with smaller annotated datasets is typical in clinical NLP and therefore, ensuring that deep learning models perform well is crucial for the models to be used in real-world applications. A widely adopted approach is fine-tuning existing Pre-trained Language Models (PLMs), but these attempts fall short when the training dataset contains only a few annotated samples. Few-Shot Learning (FSL) has recently been investigated to tackle this problem. Siamese Neural Network (SNN) has been widely utilized as an FSL approach in computer vision, but has not been studied well in NLP. Furthermore, the literature on its applications in clinical domains is scarce. In this paper, we propose two SNN-based FSL approaches for clinical NLP, including Pre-Trained SNN (PT-SNN) and SNN with Second-Order Embeddings (SOE-SNN). We evaluated the proposed approaches on two clinical tasks, namely clinical text classification and clinical named entity recognition. We tested three few-shot settings including 4-shot, 8-shot, and 16-shot learning. Both clinical NLP tasks were benchmarked using three PLMs, including BERT,BioBERT, and BioClinicalBERT. The experimental results verified the effectiveness of the proposed SNN-based FSL approaches in both NLP tasks.
