Dataset Information

Mapping of Crowdsourcing in Health: Systematic Review.

ABSTRACT: BACKGROUND:Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care). OBJECTIVE:This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsourced tasks used. We also describe the logistics of crowdsourcing and the characteristics of crowd workers. METHODS:MEDLINE, EMBASE, and ClinicalTrials.gov were searched for available reports from inception to March 30, 2016, with no restriction on language or publication status. RESULTS:We identified 202 relevant studies that used crowdsourcing, including 9 randomized controlled trials, of which only one had posted results at ClinicalTrials.gov. Crowdsourcing was used in health promotion (91/202, 45.0%), research (73/202, 36.1%), and care (38/202, 18.8%). The 4 most frequent areas of application were public health (67/202, 33.2%), psychiatry (32/202, 15.8%), surgery (22/202, 10.9%), and oncology (14/202, 6.9%). Half of the reports (99/202, 49.0%) referred to data processing, 34.6% (70/202) referred to surveying, 10.4% (21/202) referred to surveillance or monitoring, and 5.9% (12/202) referred to problem-solving. Labor market platforms (eg, Amazon Mechanical Turk) were used in most studies (190/202, 94%). The crowd workers' characteristics were poorly reported, and crowdsourcing logistics were missing from two-thirds of the reports. When reported, the median size of the crowd was 424 (first and third quartiles: 167-802); crowd workers' median age was 34 years (32-36). Crowd workers were mainly recruited nationally, particularly in the United States. For many studies (58.9%, 119/202), previous experience in crowdsourcing was required, and passing a qualification test or training was seldom needed (11.9% of studies; 24/202). For half of the studies, monetary incentives were mentioned, with mainly less than US $1 to perform the task. The time needed to perform the task was mostly less than 10 min (58.9% of studies; 119/202). Data quality validation was used in 54/202 studies (26.7%), mainly by attention check questions or by replicating the task with several crowd workers. CONCLUSIONS:The use of crowdsourcing, which allows access to a large pool of participants as well as saving time in data collection, lowering costs, and speeding up innovations, is increasing in health promotion, research, and care. However, the description of crowdsourcing logistics and crowd workers' characteristics is frequently missing in study reports and needs to be precisely reported to better interpret the study findings and replicate them.

SUBMITTER: Crequit P

PROVIDER: S-EPMC5974463 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Mapping of Crowdsourcing in Health: Systematic Review.

Créquit Perrine P Mansouri Ghizlène G Benchoufi Mehdi M Vivot Alexandre A Ravaud Philippe P

Journal of medical Internet research 20180515 5

<h4>Background</h4>Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care).<h4>Objective</h4>This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsou ...[more]

PMID: 29764795

Similar Datasets

Project description:BACKGROUND:Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health. METHODS:We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach. RESULTS:We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled. CONCLUSIONS:Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine. TRIAL REGISTRATION:PROSPERO: CRD42017052835. December 27, 2016.

Project description:BackgroundFloods are the most common type of global natural disaster. Floods have a negative impact on mental health. Comprehensive evaluation and review of the literature are lacking.ObjectiveTo systematically map and review available scientific evidence on mental health impacts of floods caused by extended periods of heavy rain in river catchments.MethodsWe performed a systematic mapping review of published scientific literature in five languages for mixed studies on floods and mental health. PUBMED and Web of Science were searched to identify all relevant articles from 1994 to May 2014 (no restrictions).ResultsThe electronic search strategy identified 1331 potentially relevant papers. Finally, 83 papers met the inclusion criteria. Four broad areas are identified: i) the main mental health disorders-post-traumatic stress disorder, depression and anxiety; ii] the factors associated with mental health among those affected by floods; iii) the narratives associated with flooding, which focuses on the long-term impacts of flooding on mental health as a consequence of the secondary stressors; and iv) the management actions identified. The quantitative and qualitative studies have consistent findings. However, very few studies have used mixed methods to quantify the size of the mental health burden as well as exploration of in-depth narratives. Methodological limitations include control of potential confounders and short-term follow up.LimitationsFloods following extreme events were excluded from our review.ConclusionsAlthough the level of exposure to floods has been systematically associated with mental health problems, the paucity of longitudinal studies and lack of confounding controls precludes strong conclusions.ImplicationsWe recommend that future research in this area include mixed-method studies that are purposefully designed, using more rigorous methods. Studies should also focus on vulnerable groups and include analyses of policy and practical responses.

Project description:Crowdsourcing is an interactive process that has a group of individuals attempt to solve all or part of a problem and then share solutions with the public. Crowdsourcing is increasingly used to enhance training through developing learning materials and promoting mentorship. This scoping review aims to assess the literature on crowdsourcing for training in public health. We searched five medical and public health research databases using terms related to crowdsourcing and training. For this review, the concept of crowdsourcing included open calls, designathons, and other activities. We used a PRISMA checklist for scoping reviews. Each full-text was assessed by two independent reviewers. We identified 4,071 citations, and 74 studies were included in the scoping review. This included one study in a low-income country, 15 studies in middle-income countries, 35 studies in high-income countries, and 11 studies conducted in multiple countries of varying income levels (the country income level for 12 studies could not be ascertained). Nine studies used open calls, 35 used a hackathon, designathon or other "a-thon" event, and 30 used other crowdsourcing methods, such as citizen science programs and online creation platforms. In terms of crowdsourcing purpose, studies used crowdsourcing to educate participants (20 studies), develop learning materials (17 studies), enhance mentorship (13 studies) and identify trainees (9 studies). Fifteen studies used crowdsourcing for more than one training purpose. Thirty-four studies were done in-person, 31 were conducted virtually and nine used both meeting options for their crowdsourcing events. Seventeen studies generated open access materials. Our review found that crowdsourcing has been increasingly used to support public health training. This participatory approach can be a useful tool for training in a variety of settings and populations. Future research should investigate the impact of crowdsourcing on training outcomes.

Project description:BackgroundCrowdsourcing engages the help of large numbers of people in tasks, activities or projects, usually via the internet. One application of crowdsourcing is the screening of citations for inclusion in a systematic review. There is evidence that a 'Crowd' of non-specialists can reliably identify quantitative studies, such as randomized controlled trials, through the assessment of study titles and abstracts. In this feasibility study, we investigated crowd performance of an online, topic-based citation-screening task, assessing titles and abstracts for inclusion in a single mixed-studies systematic review.MethodsThis study was embedded within a mixed studies systematic review of maternity care, exploring the effects of training healthcare professionals in intrapartum cardiotocography. Citation-screening was undertaken via Cochrane Crowd, an online citizen science platform enabling volunteers to contribute to a range of tasks identifying evidence in health and healthcare. Contributors were recruited from users registered with Cochrane Crowd. Following completion of task-specific online training, the crowd and the review team independently screened 9546 titles and abstracts. The screening task was subsequently repeated with a new crowd following minor changes to the crowd agreement algorithm based on findings from the first screening task. We assessed the crowd decisions against the review team categorizations (the 'gold standard'), measuring sensitivity, specificity, time and task engagement.ResultsSeventy-eight crowd contributors completed the first screening task. Sensitivity (the crowd's ability to correctly identify studies included within the review) was 84% (N = 42/50), and specificity (the crowd's ability to correctly identify excluded studies) was 99% (N = 9373/9493). Task completion was 33 h for the crowd and 410 h for the review team; mean time to classify each record was 6.06 s for each crowd participant and 3.96 s for review team members. Replicating this task with 85 new contributors and an altered agreement algorithm found 94% sensitivity (N = 48/50) and 98% specificity (N = 9348/9493). Contributors reported positive experiences of the task.ConclusionIt might be feasible to recruit and train a crowd to accurately perform topic-based citation-screening for mixed studies systematic reviews, though resource expended on the necessary customised training required should be factored in. In the face of long review production times, crowd screening may enable a more time-efficient conduct of reviews, with minimal reduction of citation-screening accuracy, but further research is needed.

Project description:BackgroundThe World Health Organisation framed responsiveness, fair financing and equity as intrinsic goals of health systems. However, of the three, responsiveness received significantly less attention. Responsiveness is essential to strengthen systems' functioning; provide equitable and accountable services; and to protect the rights of citizens. There is an urgency to make systems more responsive, but our understanding of responsiveness is limited. We therefore sought to map existing evidence on health system responsiveness.MethodsA mixed method systemized evidence mapping review was conducted. We searched PubMed, EbscoHost, and Google Scholar. Published and grey literature; conceptual and empirical publications; published between 2000 and 2020 and English language texts were included. We screened titles and abstracts of 1119 publications and 870 full texts.ResultsSix hundred twenty-one publications were included in the review. Evidence mapping shows substantially more publications between 2011 and 2020 (n = 462/621) than earlier periods. Most of the publications were from Europe (n = 139), with more publications relating to High Income Countries (n = 241) than Low-to-Middle Income Countries (n = 217). Most were empirical studies (n = 424/621) utilized quantitative methodologies (n = 232), while qualitative (n = 127) and mixed methods (n = 63) were more rare. Thematic analysis revealed eight primary conceptualizations of 'health system responsiveness', which can be fitted into three dominant categorizations: 1) unidirectional user-service interface; 2) responsiveness as feedback loops between users and the health system; and 3) responsiveness as accountability between public and the system.ConclusionsThis evidence map shows a substantial body of available literature on health system responsiveness, but also reveals evidential gaps requiring further development, including: a clear definition and body of theory of responsiveness; the implementation and effectiveness of feedback loops; the systems responses to this feedback; context-specific mechanism-implementation experiences, particularly, of LMIC and fragile-and conflict affected states; and responsiveness as it relates to health equity, minority and vulnerable populations. Theoretical development is required, we suggest separating ideas of services and systems responsiveness, applying a stronger systems lens in future work. Further agenda-setting and resourcing of bridging work on health system responsiveness is suggested.

Project description:BackgroundThe global literature on the links between climate change and human health is large, increasing exponentially, and it is no longer feasible to collate and synthesise using traditional systematic evidence mapping approaches. We aimed to use machine learning methods to systematically synthesise an evidence base on climate change and human health.MethodsWe used supervised machine learning and other natural language processing methods (topic modelling and geoparsing) to systematically identify and map the scientific literature on climate change and health published between Jan 1, 2013, and April 9, 2020. Only literature indexed in English were included. We searched Web of Science Core Collection, Scopus, and PubMed using title, abstract, and keywords only. We searched for papers including both a health component and an explicit mention of either climate change, climate variability, or climate change-relevant weather phenomena. We classified relevant publications according to the fields of climate research, climate drivers, health impact, date, and geography. We used supervised and unsupervised machine learning to identify and classify relevant articles in the field of climate and health, with outputs including evidence heat maps, geographical maps, and narrative synthesis of trends in climate health-related publications. We included empirical literature of any study design that reported on health pathways associated with climate impacts, mitigation, or adaptation.FindingsWe predict that there are 15 963 studies in the field of climate and health published between 2013 and 2019. Climate health literature is dominated by impact studies, with mitigation and adaptation responses and their co-benefits and co-risks remaining niche topics. Air quality and heat stress are the most frequently studied exposures, with all-cause mortality and infectious disease incidence being the most frequently studied health outcomes. Seasonality, extreme weather events, heat, and weather variability are the most frequently studied climate-related hazards. We found major gaps in evidence on climate health research for mental health, undernutrition, and maternal and child health. Geographically, the evidence base is dominated by studies from high-income countries and China, with scant evidence from low-income counties, which often suffer most from the health consequences of climate change.InterpretationOur findings show the importance and feasibility of using automated machine learning to comprehensively map the science on climate change and human health in the age of big literature. These can provide key inputs into global climate and health assessments. The scant evidence on climate change response options is concerning and could significantly hamper the design of evidence-based pathways to reduce the effects on health of climate change. In the post-2015 Paris Agreement era of climate solutions, we believe much more attention should be given to climate adaptation and mitigation options and their effects on human health.FundingForeign, Commonwealth & Development Office.

Project description:BackgroundHealth recommender systems (HRSs) are information retrieval systems that provide users with relevant items according to the users' needs, which can motivate and engage users to change their behavior.ObjectiveThis study aimed to identify the development and evaluation of HRSs and create an evidence map.MethodsA total of 6 databases were searched to identify HRSs reported in studies from inception up to June 30, 2022, followed by forward citation and grey literature searches. Titles, abstracts, and full texts were screened independently by 2 reviewers, with discrepancies resolved by a third reviewer, when necessary. Data extraction was performed by one reviewer and checked by a second reviewer. This review was conducted in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) statement.ResultsA total of 51 studies were included for data extraction. Recommender systems were used across different health domains, such as general health promotion, lifestyle, and generic health service. A total of 23 studies had reported the use of a combination of recommender techniques, classified as hybrid recommender systems, which are the most commonly used recommender techniques in HRSs. In the HRS design stage, only 10 of 51 (19.6%) recommender systems considered personal preferences of end users in the design or development of the system; a total of 29 studies reported the user interface of HRSs, and most HRSs worked on users' mobile interfaces, usually a mobile app. Two categories of HRS evaluations were used, and evaluations of HRSs varied greatly; 62.7% (32/51) of the studies used the offline evaluations using computational methods (no user), and 33.3% (17/51) of the studies included end users in their HRS evaluation.ConclusionsThrough this scoping review, nonmedical professionals and policy makers can visualize and better understand HRSs for future studies. The health care professionals and the end users should be encouraged to participate in the future design and development of HRSs to optimize their utility and successful implementation. Detailed evaluations of HRSs in a user-centered approach are needed in future studies.

Project description:BACKGROUND:Despite the increase in use and high expectations of digital health solutions, scientific evidence about the effectiveness of electronic health (eHealth) and other aspects such as usability and accuracy is lagging behind. eHealth solutions are complex interventions, which require a wide array of evaluation approaches that are capable of answering the many different questions that arise during the consecutive study phases of eHealth development and implementation. However, evaluators seem to struggle in choosing suitable evaluation approaches in relation to a specific study phase. OBJECTIVE:The objective of this project was to provide a structured overview of the existing eHealth evaluation approaches, with the aim of assisting eHealth evaluators in selecting a suitable approach for evaluating their eHealth solution at a specific evaluation study phase. METHODS:Three consecutive steps were followed. Step 1 was a systematic scoping review, summarizing existing eHealth evaluation approaches. Step 2 was a concept mapping study asking eHealth researchers about approaches for evaluating eHealth. In step 3, the results of step 1 and 2 were used to develop an "eHealth evaluation cycle" and subsequently compose the online "eHealth methodology guide." RESULTS:The scoping review yielded 57 articles describing 50 unique evaluation approaches. The concept mapping study questioned 43 eHealth researchers, resulting in 48 unique approaches. After removing duplicates, 75 unique evaluation approaches remained. Thereafter, an "eHealth evaluation cycle" was developed, consisting of six evaluation study phases: conceptual and planning, design, development and usability, pilot (feasibility), effectiveness (impact), uptake (implementation), and all phases. Finally, the "eHealth methodology guide" was composed by assigning the 75 evaluation approaches to the specific study phases of the "eHealth evaluation cycle." CONCLUSIONS:Seventy-five unique evaluation approaches were found in the literature and suggested by eHealth researchers, which served as content for the online "eHealth methodology guide." By assisting evaluators in selecting a suitable evaluation approach in relation to a specific study phase of the "eHealth evaluation cycle," the guide aims to enhance the quality, safety, and successful long-term implementation of novel eHealth solutions.

Dataset Information

Mapping of Crowdsourcing in Health: Systematic Review.

Publications

Mapping of Crowdsourcing in Health: Systematic Review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets