论文标题
模仿：离线和在线搜索澄清评估
MIMICS-Duo: Offline & Online Evaluation of Search Clarification
论文作者
论文摘要
提出澄清问题是一个积极的研究领域；但是，培训和评估搜索澄清方法的资源还不够。为了解决此问题，我们描述了Mimics-Duo，这是一个新的306个搜索查询数据集，具有多个澄清（总计1,034个查询贴贴对）。模仿duo包含有关澄清问题及其候选答案的细粒注释，并通过启用对搜索澄清方法的多维评估（包括在线和离线评估）来增强现有的模拟数据集。我们进行了广泛的分析，以证明离线和在线搜索澄清数据集之间的关系，并概述Mimics-Duo启用的几个研究方向。我们认为，该资源将帮助研究人员更好地了解搜索中的澄清。
Asking clarification questions is an active area of research; however, resources for training and evaluating search clarification methods are not sufficient. To address this issue, we describe MIMICS-Duo, a new freely available dataset of 306 search queries with multiple clarifications (a total of 1,034 query-clarification pairs). MIMICS-Duo contains fine-grained annotations on clarification questions and their candidate answers and enhances the existing MIMICS datasets by enabling multi-dimensional evaluation of search clarification methods, including online and offline evaluation. We conduct extensive analysis to demonstrate the relationship between offline and online search clarification datasets and outline several research directions enabled by MIMICS-Duo. We believe that this resource will help researchers better understand clarification in search.
