论文标题
自然语言推论评估归因方法的多语言视角
A Multilingual Perspective Towards the Evaluation of Attribution Methods in Natural Language Inference
论文作者
论文摘要
归因方法的大多数评估都集中在英语。在这项工作中，我们提出了一种多种语言方法，用于评估自然语言推论（NLI）任务的归因方法，以忠诚和合理性。首先，我们引入了一种新颖的跨语性策略，以基于单词对齐方式衡量忠诚，从而消除了基于擦除的评估的缺点。然后，我们考虑了不同的输出机制和聚合方法，对归因方法进行了全面的评估。最后，我们通过基于突出的解释来增强XNLI数据集，从而提供带有亮点的多语言NLI数据集，以支持将来的EXNLP研究。我们的结果表明，归因方法最适合合理性和忠诚的方法是不同的。
Most evaluations of attribution methods focus on the English language. In this work, we present a multilingual approach for evaluating attribution methods for the Natural Language Inference (NLI) task in terms of faithfulness and plausibility. First, we introduce a novel cross-lingual strategy to measure faithfulness based on word alignments, which eliminates the drawbacks of erasure-based evaluations.We then perform a comprehensive evaluation of attribution methods, considering different output mechanisms and aggregation methods. Finally, we augment the XNLI dataset with highlight-based explanations, providing a multilingual NLI dataset with highlights, to support future exNLP studies. Our results show that attribution methods performing best for plausibility and faithfulness are different.
