论文标题
通过基于能量的重新排列改善抽象性汇总
Improving abstractive summarization with energy-based re-ranking
论文作者
论文摘要
当前的抽象性摘要系统提出了重要的弱点，这些弱点阻止了它们在现实世界中的部署，例如省略相关信息和产生事实不一致（也称为幻觉）。同时，最近已经提出，自动评估指标（例如CTC得分）与人类判断的相关性比传统的词汇范围超过的指标（例如Ruge）具有更高的相关性。在这项工作中，我们打算通过利用摘要指标的最新进展来结束循环，以创建质量吸引的抽象摘要。也就是说，我们提出了一个基于能量的模型，该模型学会根据这些指标的一个或一个组合重新排列摘要。我们使用几个指标来训练我们的基于能量的重新疗程，并表明它始终如一地改善了预测的摘要所获得的分数。尽管如此，人类评估结果表明，重新排列的方法应谨慎使用高度抽象的摘要，因为可用的指标尚不足够可靠。
Current abstractive summarization systems present important weaknesses which prevent their deployment in real-world applications, such as the omission of relevant information and the generation of factual inconsistencies (also known as hallucinations). At the same time, automatic evaluation metrics such as CTC scores have been recently proposed that exhibit a higher correlation with human judgments than traditional lexical-overlap metrics such as ROUGE. In this work, we intend to close the loop by leveraging the recent advances in summarization metrics to create quality-aware abstractive summarizers. Namely, we propose an energy-based model that learns to re-rank summaries according to one or a combination of these metrics. We experiment using several metrics to train our energy-based re-ranker and show that it consistently improves the scores achieved by the predicted summaries. Nonetheless, human evaluation results show that the re-ranking approach should be used with care for highly abstractive summaries, as the available metrics are not yet sufficiently reliable for this purpose.
