Dataset Information

Producing knowledge by admitting ignorance: Enhancing data quality through an "I don't know" option in citizen science.

ABSTRACT: The "noisy labeler problem" in crowdsourced data has attracted great attention in recent years, with important ramifications in citizen science, where non-experts must produce high-quality data. Particularly relevant to citizen science is dynamic task allocation, in which the level of agreement among labelers can be progressively updated through the information-theoretic notion of entropy. Under dynamic task allocation, we hypothesized that providing volunteers with an "I don't know" option would contribute to enhancing data quality, by introducing further, useful information about the level of agreement among volunteers. We investigated the influence of an "I don't know" option on the data quality in a citizen science project that entailed classifying the image of a highly polluted canal into "threat" or "no threat" to the environment. Our results show that an "I don't know" option can enhance accuracy, compared to the case without the option; such an improvement mostly affects the true negative rather than the true positive rate. In an information-theoretic sense, these seemingly meaningless blank votes constitute a meaningful piece of information to help enhance accuracy of data in citizen science.

SUBMITTER: Torre M

PROVIDER: S-EPMC6392254 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Producing knowledge by admitting ignorance: Enhancing data quality through an "I don't know" option in citizen science.

Torre Marina M Nakayama Shinnosuke S Tolbert Tyrone J TJ Porfiri Maurizio M

PloS one 20190227 2

The "noisy labeler problem" in crowdsourced data has attracted great attention in recent years, with important ramifications in citizen science, where non-experts must produce high-quality data. Particularly relevant to citizen science is dynamic task allocation, in which the level of agreement among labelers can be progressively updated through the information-theoretic notion of entropy. Under dynamic task allocation, we hypothesized that providing volunteers with an "I don't know" option woul ...[more]

PMID: 30811452

Similar Datasets

Project description:BACKGROUND:Disaster citizen science, or the use of scientific principles and methods by "non-professional" scientists or volunteers, may be a promising way to enhance public health emergency preparedness (PHEP) and build community resilience. However, little research has focused on understanding this emerging field and its implications for PHEP. To address research gaps, this paper: (1) assesses the state of disaster citizen science by developing an inventory of disaster citizen science projects; (2) identifies different models of disaster citizen science; and (3) assesses their relevance for PHEP. METHODS:We searched the English-language peer-reviewed and grey literature for disaster citizen science projects with no time period specified. Following searches, a team of three reviewers applied inclusion/exclusion criteria that defined eligible disasters and citizen science activities. Reviewers extracted the following elements from each project: project name and description; lead and partner entities; geographic setting; start and end dates; type of disaster; disaster phase; citizen science model; and technologies used. RESULTS:A final set of 209 projects, covering the time period 1953-2017, were included in the inventory. Projects were classified across five citizen science models: distributed or volunteer sensing (n =?19; 9%); contributory (n =?98; 47%); distributed intelligence (n =?52; 25%); collaborative research (n =?32; 15%); and collegial research (n =?8; 4%). Overall, projects were conducted across all disaster phases and most frequently for earthquakes, floods, and hurricanes. Although activities occurred globally, 40% of projects were set in the U.S. Academic, government, technology, and advocacy organizations were the most prevalent lead entities. Although a range of technologies were used, 77% of projects (n =?161) required an internet-connected device. These characteristics varied across citizen science models revealing important implications for applications of disaster citizen science, enhancement of disaster response capabilities, and sustainability of activities over time. CONCLUSIONS:By increasing engagement in research, disaster citizen science may empower communities to take collective action, improve system response capabilities, and generate relevant data to mitigate adverse health impacts. The project inventory established a baseline for future research to capitalize on opportunities, address limitations, and help disaster citizen science achieve its potential.

Project description:BackgroundBiodiversity is being lost rapidly and its conservation is thus one of the most urgent tasks today. For biodiversity conservation to be successful, the public needs to gain an awareness and understanding of biodiversity and its importance. Moreover, species experts are needed who have the skills necessary for identifying and recording biodiversity. Previous research showed that citizen science projects can contribute to educating the public about biodiversity. However, it is still unclear how project characteristics connect to participants' knowledge and skills and how citizen science projects should be designed if they are to foster participants' learning.AimWe aimed to investigate specific characteristics of biodiversity citizen science projects that could potentially influence participants' learning. We explored the following project characteristics from both the project coordinators' and the participants' perspectives: information and training provided to participants, social interaction among participants, contact between participants and staff, and feedback and recognition provided to participants.Methods and resultsIn order to examine the extent to which these project characteristics are connected to participants' gains in knowledge and skills, we conducted a comprehensive study across 48 biodiversity citizen science projects in Europe and Australia. We found that participants' perceived gains in knowledge and skills were significantly related to the five project characteristics as reported by the participants: information received by the participants, training received by the participants, social interaction among participants, contact between participants and staff, and feedback and recognition received by the participants.ConclusionWe conclude that by deliberately designing citizen science projects to include features such as interaction and feedback, these projects could achieve higher learning outcomes for the participants. Thereby, suitable modes of communication between projects and their participants are crucial. We provide specific suggestions for the design of biodiversity citizen science projects and for future research on project characteristics and participant outcomes.

Dataset Information

Producing knowledge by admitting ignorance: Enhancing data quality through an "I don't know" option in citizen science.

Publications

Producing knowledge by admitting ignorance: Enhancing data quality through an "I don't know" option in citizen science.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets