The Validity and Credibility of Crowdsourced Fact-checking

Abstract

Demand of fact-checking has increased together with the amount of information. However, fact-checking is often neglected due to lack of time and labor source. Also, development of digital technology has contributed to the acceleration of information spread. As online information is overabundant, it is difficult for professional journalists to fact-check all information before publishing it. Many researchers are developing algorithms for fact-checking, but they are not practical yet. In order to assist those deficiencies of fact-checking algorithms, collective intelligence was considered as an alternative method. With the help from the public, journalists collect, classify, and analyze data, and also widen their perspectives. This research aims to examine the validity of crowdsourced fact-checking and its credibility level compared to professional journalism fact-checking results. Also, the interface elements to enhance credibility of crowdsourced fact-checking results were observed. The results show that crowdsourced fact-checking process is promising except for ambiguous and partially true claims. The Mechanical Turk workers provided their deliberate opinions and critical evidence. Their rationale implies the possibility of public discussion and their capability of narrowing down broad statements to verifiable sentences. The credibility level of traditional journalism fact-checking was generally higher than crowdsourced results but for certain categories crowdsourced condition had a higher credibility level due to the features of social media. Moreover, the reputation of the user was more influential to the credibility than social information disclosure level. The findings from current research implies the future design of fact-checking platform and how could the current fact-checking algorithms could benefit from the collective intelligence.