论文标题
关于谓词复杂性在众包分类任务中的影响
On the impact of predicate complexity in crowdsourced classification tasks
论文作者
论文摘要
本文探讨并提供有关众包任务设计中特定和相关问题的指导：如何制定用于对一组项目进行分类的复杂问题。在微任务市场中，分类仍然是最受欢迎的任务之一。我们将工作置于信息检索和多培养基分类的背景下，即根据一组条件对一组项目进行分类。我们的实验涵盖了广泛的任务和领域，也单独考虑与机器学习分类器同时考虑群众。我们提供了经验证据表明，所产生的分类性能如何受到不同谓词表述策略的影响，强调了谓词表述作为众包中的任务设计维度的重要性。
This paper explores and offers guidance on a specific and relevant problem in task design for crowdsourcing: how to formulate a complex question used to classify a set of items. In micro-task markets, classification is still among the most popular tasks. We situate our work in the context of information retrieval and multi-predicate classification, i.e., classifying a set of items based on a set of conditions. Our experiments cover a wide range of tasks and domains, and also consider crowd workers alone and in tandem with machine learning classifiers. We provide empirical evidence into how the resulting classification performance is affected by different predicate formulation strategies, emphasizing the importance of predicate formulation as a task design dimension in crowdsourcing.
