Title: A crowdsourced system for user studies in information extraction

Authors: Zohreh Khojasteh-Ghamari

Addresses: Department of Environmental Sciences, Informatics and Statistics, Ca' Foscari University of Venice, Via Torino, 155, 30172 Venezia Mestre, Italy

Abstract: In this paper, from an entity linking (EL) system, we take a set of tweets, where some subsequence of words is annotated with possible meaning/entities and these entities are linked with several Wikipedia pages. We propose a model using crowdsourcing to disambiguate and decide about the accurate Wikipedia page that must be linked with a definite word/spot. We discuss about importance of crowdsourcing and compare different crowdsourcing systems and at the end, introduce crowdflower. We discuss about the crowdflower features in particular. Finally, we analyse output reports of the crowdflower and present a novel approach to select the reliable results. In summary, our observations show that reliable results have a confidence rate over 0.5.

Keywords: crowdsourcing; information extraction; data mining.

DOI: 10.1504/IJKESDP.2017.089506

International Journal of Knowledge Engineering and Soft Data Paradigms, 2017 Vol.6 No.1, pp.44 - 51

Received: 08 Jan 2016
Accepted: 09 May 2017
Published online: 29 Jan 2018 *