Dataset Information

Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

ABSTRACT: Objectives:Usable tools to support individual primary care clinicians in their diagnostic processes could help to reduce preventable harm from diagnostic errors. We conducted a formative study with primary care providers to identify key requisites to optimize the acceptability of 1 online collective intelligence platform (Human Diagnosis Project; Human Dx). Materials and Methods:We conducted semistructured interviews with practicing primary care clinicians in a sample of the US community-based clinics to examine the acceptability and early usability of the collective intelligence online platform using standardized clinical cases and real-world clinical cases from the participants' own practice. We used an integrated inductive-deductive qualitative analysis approach to analyze the interview transcripts. Results and Discussion:Perceived usefulness, perceived accuracy, quality assurance, trust, and ease of use emerged as essential domains of acceptability required for providers to use a collective intelligence tool in clinical practice. Participants conveyed that the collective opinion should: (1) contribute to their clinical reasoning, (2) boost their confidence, (3) be generated in a timely manner, and (4) be relevant to their clinical settings and use cases. Trust in the technology platform and the clinical accuracy of its collective intelligence output emerged as an incontrovertible requirement for user acceptance and engagement. Conclusion:We documented key requisites to building a collective intelligence technology platform that is trustworthy, useful, and acceptable to target end users for assistance in the diagnostic process. These key lessons may be applicable to other provider-facing decision support platforms.

SUBMITTER: Fontil V

PROVIDER: S-EPMC6952011 | biostudies-literature | 2019 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

Fontil Valy V Radcliffe Kate K Lyson Helena C HC Ratanawongsa Neda N Lyles Courtney C Tuot Delphine D Yuen Kaeli K Sarkar Urmimala U

JAMIA open 20190201 1

<h4>Objectives</h4>Usable tools to support individual primary care clinicians in their diagnostic processes could help to reduce preventable harm from diagnostic errors. We conducted a formative study with primary care providers to identify key requisites to optimize the acceptability of 1 online collective intelligence platform (Human Diagnosis Project; Human Dx).<h4>Materials and methods</h4>We conducted semistructured interviews with practicing primary care clinicians in a sample of the US co ...[more]

PMID: 31984344

Similar Datasets

Project description:BACKGROUND:The diagnosis of pigmented skin lesion is error prone and requires domain-specific expertise, which is not readily available in many parts of the world. Collective intelligence could potentially decrease the error rates of nonexperts. OBJECTIVE:The aim of this study was to evaluate the feasibility and impact of collective intelligence for the detection of skin cancer. METHODS:We created a gamified study platform on a stack of established Web technologies and presented 4216 dermatoscopic images of the most common benign and malignant pigmented skin lesions to 1245 human raters with different levels of experience. Raters were recruited via scientific meetings, mailing lists, and social media posts. Education was self-declared, and domain-specific experience was tested by screening tests. In the target test, the readers had to assign 30 dermatoscopic images to 1 of the 7 disease categories. The readers could repeat the test with different lesions at their own discretion. Collective human intelligence was achieved by sampling answers from multiple readers. The disease category with most votes was regarded as the collective vote per image. RESULTS:We collected 111,019 single ratings, with a mean of 25.2 (SD 18.5) ratings per image. As single raters, nonexperts achieved a lower mean accuracy (58.6%) than experts (68.4%; mean difference=-9.4%; 95% CI -10.74% to -8.1%; P<.001). Collectives of nonexperts achieved higher accuracies than single raters, and the improvement increased with the size of the collective. A collective of 4 nonexperts surpassed single nonexperts in accuracy by 6.3% (95% CI 6.1% to 6.6%; P<.001). The accuracy of a collective of 8 nonexperts was 9.7% higher (95% CI 9.5% to 10.29%; P<.001) than that of single nonexperts, an improvement similar to single experts (P=.73). The sensitivity for malignant images increased for nonexperts (66.3% to 77.6%) and experts (64.6% to 79.4%) for answers given faster than the intrarater mean. CONCLUSIONS:A high number of raters can be attracted by elements of gamification and Web-based marketing via mailing lists and social media. Nonexperts increase their accuracy to expert level when acting as a collective, and faster answers correspond to higher accuracy. This information could be useful in a teledermatology setting.

Project description:BackgroundU.S. hospitals and dialysis centers are penalized for 30-day hospital readmissions of dialysis patients, despite little infrastructure to facilitate care transitions between these settings. We are developing a third-party web-based information exchange platform, DialysisConnect, to enable clinicians to view and exchange information about dialysis patients during admission, hospitalization, and discharge. This health information technology solution could serve as a flexible and relatively affordable solution for dialysis facilities and hospitals across the nation who are seeking to serve as true partners in the improved care of dialysis patients. The purpose of this study was to evaluate the perceived coherence of DialysisConnect to key clinical stakeholders, to prepare messaging for implementation.MethodsAs part of a hybrid effectiveness-implementation study guided by Normalization Process Theory, we collected data on stakeholder perceptions of continuity of care for patients receiving maintenance dialysis and a DialysisConnect prototype before completing development and piloting the system. We conducted four focus groups with stakeholders from one academic hospital and associated dialysis centers [hospitalists (n = 5), hospital staff (social workers, nurses, pharmacists; n = 9), nephrologists (n = 7), and dialysis clinic staff (social workers, nurses; n = 10)]. Transcriptions were analyzed thematically within each component of the construct of coherence (differentiation, communal specification, individual specification, and internalization).ResultsParticipants differentiated DialysisConnect from usual care variously as an information dashboard, a quick-exchange communication channel, and improved discharge information delivery; some could not differentiate it in terms of workflow. The purpose of DialysisConnect (communal specification) was viewed as fully coherent only for communicating outside of the same healthcare system. Current system workarounds were acknowledged as deterrents for practice change. All groups delegated DialysisConnect tasks (individual specification) to personnel besides themselves. Partial internalization of DialysisConnect was achieved only by dialysis clinic staff, based on experience with similar technology.ConclusionsImplementing DialysisConnect for clinical users in both settings will require presenting a composite picture of current communication processes from all stakeholder groups to correct single-group misunderstandings, as well as providing data about care transitions communication beyond the local context to ease resistance to practice change.

Project description:BackgroundDiagnostic error in ambulatory care, a frequent cause of preventable harm, may be mitigated using the collective intelligence of multiple clinicians. The National Academy of Medicine has identified enhanced clinician collaboration and digital tools as a means to improve the diagnostic process.ObjectiveThis study aims to assess the efficacy of a collective intelligence output to improve diagnostic confidence and accuracy in ambulatory care cases (from primary care and urgent care clinic visits) with diagnostic uncertainty.MethodsThis is a pragmatic randomized controlled trial of using collective intelligence in cases with diagnostic uncertainty from clinicians at primary care and urgent care clinics in 2 health care systems in San Francisco. Real-life cases, identified for having an element of diagnostic uncertainty, will be entered into a collective intelligence digital platform to acquire collective intelligence from at least 5 clinician contributors on the platform. Cases will be randomized to an intervention group (where clinicians will view the collective intelligence output) or control (where clinicians will not view the collective intelligence output). Clinicians will complete a postvisit questionnaire that assesses their diagnostic confidence for each case; in the intervention cases, clinicians will complete the questionnaire after reviewing the collective intelligence output for the case. Using logistic regression accounting for clinician clustering, we will compare the primary outcome of diagnostic confidence and the secondary outcome of time with diagnosis (the time it takes for a clinician to reach a diagnosis), for intervention versus control cases. We will also assess the usability and satisfaction with the digital tool using measures adapted from the Technology Acceptance Model and Net Promoter Score.ResultsWe have recruited 32 out of our recruitment goal of 33 participants. This study is funded until May 2020 and is approved by the University of California San Francisco Institutional Review Board until January 2020. We have completed data collection as of June 2019 and will complete our proposed analysis by December 2019.ConclusionsThis study will determine if the use of a digital platform for collective intelligence is acceptable, useful, and efficacious in improving diagnostic confidence and accuracy in outpatient cases with diagnostic uncertainty. If shown to be valuable in improving clinicians' diagnostic process, this type of digital tool may be one of the first innovations used for reducing diagnostic errors in outpatient care. The findings of this study may provide a path forward for improving the diagnostic process.International registered report identifier (irrid)DERR1-10.2196/13151.

Project description:BACKGROUND:Innovative ways of planning and conducting research have emerged recently, based on the concept of collective intelligence. Collective intelligence is defined as shared intelligence emerging when people are mobilized within or outside an organization to work on a specific task that could result in more innovative outcomes than those when individuals work alone. Crowdsourcing is defined as "the act of taking a job traditionally performed by a designated agent and outsourcing it to an undefined, generally large group of people in the form of an open call." OBJECTIVE:This qualitative study aimed to identify the barriers to mobilizing collective intelligence and ways to overcome these barriers and provide good practice advice for planning and conducting collective intelligence projects across different research disciplines. METHODS:We conducted a multinational online open-ended question survey and semistructured audio-recorded interviews with a purposive sample of researchers who had experience in running collective intelligence projects. The questionnaires had an interactive component, enabling respondents to rate and comment on the advice of their fellow respondents. Data were analyzed thematically, drawing on the framework method. RESULTS:A total of 82 respondents from various research fields participated in the survey (n=65) or interview (n=17). The main barriers identified were the lack of evidence-based guidelines for implementing collective intelligence, complexity in recruiting and engaging the community, and difficulties in disseminating the results of collective intelligence projects. We drew on respondents' experience to provide tips and good practice advice for governance, planning, and conducting collective intelligence projects. Respondents particularly suggested establishing a diverse coordination team to plan and manage collective intelligence projects and setting up common rules of governance for participants in projects. In project planning, respondents provided advice on identifying research problems that could be answered by collective intelligence and identifying communities of participants. They shared tips on preparing the task and interface and organizing communication activities to recruit and engage participants. CONCLUSIONS:Mobilizing collective intelligence through crowdsourcing is an innovative method to increase research efficiency, although there are several barriers to its implementation. We present good practice advice from researchers with experience of collective intelligence across different disciplines to overcome barriers to mobilizing collective intelligence.

Dataset Information

Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

Publications

Testing and improving the acceptability of a web-based platform for collective intelligence to improve diagnostic accuracy in primary care clinics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets