Dataset Information

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

ABSTRACT:

Background

Using malpractice claims cases as vignettes is a promising approach for improving clinical reasoning education (CRE), as malpractice claims can provide a variety of content- and context-rich examples. However, the effect on learning of adding information about a malpractice claim, which may evoke a deeper emotional response, is not yet clear. This study examined whether knowing that a diagnostic error resulted in a malpractice claim affects diagnostic accuracy and self-reported confidence in the diagnosis of future cases. Moreover, suitability of using erroneous cases with and without a malpractice claim for CRE, as judged by participants, was evaluated.

Methods

In the first session of this two-phased, within-subjects experiment, 81 first-year residents of general practice (GP) were exposed to both erroneous cases with (M) and erroneous cases without (NM) malpractice claim information, derived from a malpractice claims database. Participants rated suitability of the cases for CRE on a five-point Likert scale. In the second session, one week later, participants solved four different cases with the same diagnoses. Diagnostic accuracy was measured with three questions, scored on a 0-1 scale: (1) What is your next step? (2) What is your differential diagnosis? (3) What is your most probable diagnosis and what is your level of certainty on this? Both subjective suitability and diagnostic accuracy scores were compared between the versions (M and NM) using repeated measures ANOVA.

Results

There were no differences in diagnostic accuracy parameters (M vs. NM next step: 0.79 vs. 0.77, p = 0.505; differential diagnosis 0.68 vs. 0.75, p = 0.072; most probable diagnosis 0.52 vs. 0.57, p = 0.216) and self-reported confidence (53.7% vs. 55.8% p = 0.390) of diagnoses previously seen with or without malpractice claim information. Subjective suitability- and complexity scores for the two versions were similar (suitability: 3.68 vs. 3.84, p = 0.568; complexity 3.71 vs. 3.88, p = 0.218) and significantly increased for higher education levels for both versions.

Conclusion

The similar diagnostic accuracy rates between cases studied with or without malpractice claim information suggests both versions are equally effective for CRE in GP training. Residents judged both case versions to be similarly suitable for CRE; both were considered more suitable for advanced than for novice learners.

SUBMITTER: van Sassen C

PROVIDER: S-EPMC10294315 | biostudies-literature | 2023 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

van Sassen Charlotte C Mamede Silvia S Bos Michiel M van den Broek Walter W Bindels Patrick P Zwaan Laura L

BMC medical education 20230626 1

<h4>Background</h4>Using malpractice claims cases as vignettes is a promising approach for improving clinical reasoning education (CRE), as malpractice claims can provide a variety of content- and context-rich examples. However, the effect on learning of adding information about a malpractice claim, which may evoke a deeper emotional response, is not yet clear. This study examined whether knowing that a diagnostic error resulted in a malpractice claim affects diagnostic accuracy and self-reporte ...[more]

PMID: 37365590

Similar Datasets

Project description:ObjectiveClinicians may hesitate to advocate for autopsies out of concern for increased malpractice risk if the pathological findings at time of death differ from the clinical findings. We aimed to understand the impact of autopsy findings on malpractice claim outcomes.MethodsClosed malpractice claims with loss dates between 1995 and 2015 involving death related to inpatient care at 3 Harvard Medical School hospitals were extracted from a captive malpractice insurer's database. These claims were linked to patients' electronic health records and their autopsy reports. Using the Goldman classification system, 2 physician reviewers blinded to claim outcome determined whether there was major, minor, or no discordance between the final clinical diagnoses and pathologic diagnoses. Claims were compared depending on whether an autopsy was performed and whether there was major versus minor/no clinical-pathologic discordance. Primary outcomes included percentage of claims paid through settlement or plaintiff verdict and the amount of indemnity paid, inflation adjusted.ResultsOf 293 malpractice claims related to an inpatient death that could be linked to patients' electronic health records, 89 claims (30%) had an autopsy performed by either the hospital or medical examiner. The most common claim allegation was an issue with clinician diagnosis, which was statistically less common in the autopsy group (18% versus 38%, P = 0.001). There was no difference in percentage of claims paid whether an autopsy was performed or not (42% versus 41%, P = 0.90) and no difference in median indemnity of paid claims after adjusting for number of defendants ($1,180,537 versus $906,518, P = 0.15). Thirty-one percent of claims with hospital autopsies performed demonstrated major discordance between autopsy and clinical findings. Claims with major clinical-pathologic discordance also did not have a statistically significant difference in percentage paid (44% versus 41%, P > 0.99) or amount paid ($895,954 versus $1,494,120, P = 0.10) compared with claims with minor or no discordance.ConclusionsAlthough multiple factors determine malpractice claim outcome, in this cohort, claims in which an autopsy was performed did not result in more paid outcomes, even when there was major discordance between clinical and pathologic diagnoses.

Project description:BackgroundPhysicians often select clinical management strategies not strongly supported by evidence or guidelines. Our objective was to examine the likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians using clinical vignettes of eight common medical admissions.MethodsWe conducted a cross-sectional survey using clinical vignettes of attending physicians and housestaff at one internal medicine program in New York City. Each clinical vignette included a brief clinical scenario and a varying number of clinical management strategies: diagnostic tests, consultations, and treatments, some of which had strong evidence or guideline support (Level 1 strategies) while others had limited evidence or guideline support (Level 3 strategies). Likelihood of selecting a given management strategy was assessed using Likert scales and multiple response options were used to indicate rationale(s) for selections.ResultsOur sample included 79 physicians; 68 (86%) were younger than 40 years of age, 34 (43%) were female. There were 31 attending physicians (39%) and 48 housestaff (61%) and 39 (49%) had or planned to have primarily primary care internal medicine clinical responsibilities. Overall, physicians were more likely to select Level 1 strategies "always" or "most of the time" when compared with Level 3 strategies (82% vs. 43%; p < 0.001), with wide variation across the eight medical admissions. There were no differences between attending and housestaff physician likelihood of selecting Level 3 strategies (47% vs. 45%, p = 0.36). Supportive evidence and local practice patterns were the two most common rationales behind selections; supportive evidence was cited as the most common rationale for selecting Level 1 when compared with Level 3 strategies (63% versus 30%; p < 0.001), whereas ruling out other severe conditions was cited most often for Level 3 strategies.ConclusionsFor eight common medical admissions, physicians selected more than 80% of management strategies with strong evidence or guideline support, but also selected more than 40% of strategies for which there was limited evidence or guideline support. The promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians.

Project description:PurposeGraduating medical students need broad clinical diagnostic reasoning skills that integrate learning across clinical specialties to deal with undifferentiated patient problems. The opportunity to acquire these skills may be limited during clinical placements on increasingly specialized hospital wards. We developed an intervention of regular general practitioner (GP) facilitated teaching in hospital placements to enable students to develop broad clinical diagnostic reasoning. The intervention was piloted, refined and delivered to a whole cohort of medical students at the start of their third year. This paper examines whether students perceived opportunities to improve their broad diagnostic clinical reasoning through our intervention.MethodsGP-facilitated teaching sessions were delivered weekly in hospital placements to small groups of 6–8 students for 90 mins over 6 weeks. Students practiced clinical reasoning with real patient cases that they encountered on their placements. Evaluation of learning outcomes was conducted through a student questionnaire using Likert scales with free-text boxes for additional explanation. Focus groups were conducted to gain a more in-depth understanding of student perspectives.ResultsAs high as 87% of students agreed that their broad clinical diagnostic reasoning ability had improved. Thematic analysis of the qualitative data revealed four factors supporting this improvement: practicing the hypothetico-deductive method, using real patient cases, composing student groups from different speciality placements and the breadth of the facilitators’ knowledge. Students additionally reported enhanced person-centredness in terms of understanding the patient’s perspective and journey. Students perceived that the added value of general practitioner facilitators lay in their broad knowledge base and knowledge of patient needs in the community.ConclusionOur results suggest that medical students can develop broad clinical diagnostic reasoning skills in hospital settings through regular GP-facilitated teaching. Our approach has the advantage of working within the established curricular format of hospital placements and being deliverable at scale to whole student cohorts.

Project description:BackgroundThe diagnostic accuracy of differential diagnoses generated by artificial intelligence chatbots, including ChatGPT models, for complex clinical vignettes derived from general internal medicine (GIM) department case reports is unknown.ObjectiveThis study aims to evaluate the accuracy of the differential diagnosis lists generated by both third-generation ChatGPT (ChatGPT-3.5) and fourth-generation ChatGPT (ChatGPT-4) by using case vignettes from case reports published by the Department of GIM of Dokkyo Medical University Hospital, Japan.MethodsWe searched PubMed for case reports. Upon identification, physicians selected diagnostic cases, determined the final diagnosis, and displayed them into clinical vignettes. Physicians typed the determined text with the clinical vignettes in the ChatGPT-3.5 and ChatGPT-4 prompts to generate the top 10 differential diagnoses. The ChatGPT models were not specially trained or further reinforced for this task. Three GIM physicians from other medical institutions created differential diagnosis lists by reading the same clinical vignettes. We measured the rate of correct diagnosis within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and the top diagnosis.ResultsIn total, 52 case reports were analyzed. The rates of correct diagnosis by ChatGPT-4 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 83% (43/52), 81% (42/52), and 60% (31/52), respectively. The rates of correct diagnosis by ChatGPT-3.5 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 73% (38/52), 65% (34/52), and 42% (22/52), respectively. The rates of correct diagnosis by ChatGPT-4 were comparable to those by physicians within the top 10 (43/52, 83% vs 39/52, 75%, respectively; P=.47) and within the top 5 (42/52, 81% vs 35/52, 67%, respectively; P=.18) differential diagnosis lists and top diagnosis (31/52, 60% vs 26/52, 50%, respectively; P=.43) although the difference was not significant. The ChatGPT models' diagnostic accuracy did not significantly vary based on open access status or the publication date (before 2011 vs 2022).ConclusionsThis study demonstrates the potential diagnostic accuracy of differential diagnosis lists generated using ChatGPT-3.5 and ChatGPT-4 for complex clinical vignettes from case reports published by the GIM department. The rate of correct diagnoses within the top 10 and top 5 differential diagnosis lists generated by ChatGPT-4 exceeds 80%. Although derived from a limited data set of case reports from a single department, our findings highlight the potential utility of ChatGPT-4 as a supplementary tool for physicians, particularly for those affiliated with the GIM department. Further investigations should explore the diagnostic accuracy of ChatGPT by using distinct case materials beyond its training data. Such efforts will provide a comprehensive insight into the role of artificial intelligence in enhancing clinical decision-making.

Dataset Information

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

Background

Methods

Results

Conclusion

Publications

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets