Bayesian reasoning in residents' preliminary diagnoses.
Ontology highlight
ABSTRACT:
SUBMITTER: Rottman BM
PROVIDER: S-EPMC5256430 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Representing statistical information in terms of natural frequencies rather than probabilities improves performance in Bayesian inference tasks. This beneficial effect of natural frequencies has been demonstrated in a variety of applied domains such as medicine, law, and education. Yet all the research and applications so far have been limited to situations where one dichotomous cue is used to infer which of two hypotheses is true. Real-life applications, however, often involve situations where cues (e.g., medical tests) have more than one value, where more than two hypotheses (e.g., diseases) are considered, or where more than one cue is available. In Study 1, we show that natural frequencies, compared to information stated in terms of probabilities, consistently increase the proportion of Bayesian inferences made by medical students in four conditions-three cue values, three hypotheses, two cues, or three cues-by an average of 37 percentage points. In Study 2, we show that teaching natural frequencies for simple tasks with one dichotomous cue and two hypotheses leads to a transfer of learning to complex tasks with three cue values and two cues, with a proportion of 40 and 81% correct inferences, respectively. Thus, natural frequencies facilitate Bayesian reasoning in a much broader class of situations than previously thought.
| S-EPMC4604268 | biostudies-literature
Project description:BackgroundBurnout results from excessive demands at work. Caregivers suffering from burnout show a state of emotional exhaustion, leading them to distance themselves from their patients and to become less efficient in their work. While some studies have shown a negative impact of burnout on physicians' clinical reasoning, others have failed to demonstrate any such impacts. To better understand the link between clinical reasoning and burnout, we carried out a study looking for an association between burnout and clinical reasoning in a population of general practice residents.MethodsWe conducted a cross-sectional observational study among residents in general practice in 2017 and 2019. Clinical reasoning performance was assessed using a script concordance test (SCT). The Maslach Burnout Inventory for Human Services Survey (MBI-HSS) was used to determine burnout status in both original standards of Maslach's burnout inventory manual (conventional approach) and when individuals reported high emotional exhaustion in combination with high depersonalization or low personal accomplishment compared to a norm group ("emotional exhaustion +1" approach).ResultsOne hundred ninety-nine residents were included. The participants' mean SCT score was 76.44% (95% CI: 75.77-77.10). In the conventional approach, 126 residents (63.31%) had no burnout, 37 (18.59%) had mild burnout, 23 (11.56%) had moderate burnout, and 13 (6.53%) had severe burnout. In the "exhaustion +?1" approach, 38 residents had a burnout status (19.10%). We found no significant correlation between burnout status and SCT scores either for conventional or "exhaustion +?1" approaches.ConclusionsOur data seem to indicate that burnout status has no significant impact on clinical reasoning. However, one speculation is that SCT mostly examines the clinical reasoning process's analytical dimension, whereas emotions are conventionally associated with the intuitive dimension. We think future research might aim to explore the impact of burnout on intuitive clinical reasoning processes.
| S-EPMC7792007 | biostudies-literature
Project description:IntroductionA primary aim of residency training is to develop competence in clinical reasoning. However, there are few instruments that can accurately, reliably, and efficiently assess residents' clinical decision-making ability. This study aimed to externally validate the script concordance test in emergency medicine (SCT-EM), an assessment tool designed for this purpose.MethodsUsing established methodology for the SCT-EM, we compared EM residents' performance on the SCT-EM to an expert panel of emergency physicians at three urban academic centers. We performed adjusted pairwise t-tests to compare differences between all residents and attending physicians, as well as among resident postgraduate year (PGY) levels. We tested correlation between SCT-EM and Accreditation Council for Graduate Medical Education Milestone scores using Pearson's correlation coefficients. Inter-item covariances for SCT items were calculated using Cronbach's alpha statistic.ResultsThe SCT-EM was administered to 68 residents and 13 attendings. There was a significant difference in mean scores among all groups (mean + standard deviation: PGY-1 59 + 7; PGY-2 62 + 6; PGY-3 60 + 8; PGY-4 61 + 8; 73 + 8 for attendings, p < 0.01). Post hoc pairwise comparisons demonstrated that significant difference in mean scores only occurred between each PGY level and the attendings (p < 0.01 for PGY-1 to PGY-4 vs attending group). Performance on the SCT-EM and EM Milestones was not significantly correlated (r = 0.12, p = 0.35). Internal reliability of the exam was determined using Cronbach's alpha, which was 0.67 for all examinees, and 0.89 in the expert-only group.ConclusionThe SCT-EM has limited utility in reliably assessing clinical reasoning among EM residents. Although the SCT-EM was able to differentiate clinical reasoning ability between residents and expert faculty, it did not between PGY levels, or correlate with Milestones scores. Furthermore, several limitations threaten the validity of the SCT-EM, suggesting further study is needed in more diverse settings.
| S-EPMC7390545 | biostudies-literature
Project description:Introduction:There is a need for educational resources supporting the practice and assessment of the complex processes of clinical reasoning in the inpatient setting along a continuum of physician experience levels. Methods:Using participatory design, we created a scenario-based simulation integrating diagnostic ambiguity, contextual factors, and rising patient acuity to increase complexity. Resources include an open-ended written exercise and think-aloud reflection protocol to elicit diagnostic and management reasoning and reflection on that reasoning. Descriptive statistics were used to analyze the initial implementation evaluation results. Results:Twenty physicians from multiple training stages and specialties (interns, residents, attendings, family physicians, internists, surgeons) underwent the simulated scenario. Participants engaged in clinical reasoning processes consistent with the design, considering a total of 19 differential diagnoses. Ten participants provided the correct leading diagnosis, tension pneumothorax, with an additional eight providing pneumothorax and all participants offering relevant supporting evidence. There was also good evidence of management reasoning, with all participants either performing an intervention or calling for assistance and reflecting on management plans in the think-aloud. The scenario was a reasonable approximation of clinical practice, with a mean authenticity rating of 4.15 out of 5. Finally, the scenario presented adequate challenge, with interns and residents rating it as only slightly more challenging (means of 7.83 and 7.17, respectively) than attendings (mean of 6.63 out of 10). Discussion:Despite the challenges of scenario complexity, evaluation results indicate that this resource supports the observation and analysis of diagnostic and management reasoning of diverse specialties from interns through attendings.
| S-EPMC6869982 | biostudies-literature
Project description:The belief-bias effect is one of the most-studied biases in reasoning. A recent study of the phenomenon using the signal detection theory (SDT) model called into question all theoretical accounts of belief bias by demonstrating that belief-based differences in the ability to discriminate between valid and invalid syllogisms may be an artifact stemming from the use of inappropriate linear measurement models such as analysis of variance (Dube et al., Psychological Review, 117(3), 831-863, 2010). The discrepancy between Dube et al.'s, Psychological Review, 117(3), 831-863 (2010) results and the previous three decades of work, together with former's methodological criticisms suggests the need to revisit earlier results, this time collecting confidence-rating responses. Using a hierarchical Bayesian meta-analysis, we reanalyzed a corpus of 22 confidence-rating studies (N =?993). The results indicated that extensive replications using confidence-rating data are unnecessary as the observed receiver operating characteristic functions are not systematically asymmetric. These results were subsequently corroborated by a novel experimental design based on SDT's generalized area theorem. Although the meta-analysis confirms that believability does not influence discriminability unconditionally, it also confirmed previous results that factors such as individual differences mediate the effect. The main point is that data from previous and future studies can be safely analyzed using appropriate hierarchical methods that do not require confidence ratings. More generally, our results set a new standard for analyzing data and evaluating theories in reasoning. Important methodological and theoretical considerations for future work on belief bias and related domains are discussed.
| S-EPMC6267550 | biostudies-other
Project description:OBJECTIVES:Given an aging human immunodeficiency virus (HIV) population, we aimed to determine the prevalence of HIV for long-stay residents in US nursing homes (NHs) between 2001 and 2010 and to compare characteristics and diagnoses of HIV-positive (HIV+) and negative (HIV-) residents. Also, for residents with dementia diagnoses, we compared antipsychotic (APS) medication receipt by HIV status. DESIGN:A cross-sectional comparative study. SETTING:NHs in the 14 states accounting for 75% of persons living with HIV. PARTICIPANTS:A total of 9 245 009 long-stay NH residents. MEASUREMENTS:Using Medicaid fee-for-service claims data in the years 2001 to 2010, together with Medicare resident assessment and Chronic Condition Warehouse data, we identified long-stay (more than 89?days) NH residents by HIV status and dementia presence. We examined dementia presence by age groups and APS medication receipt by younger (aged younger than 65 years) vs older (aged 65 years or older) residents, using logistic regression. RESULTS:Between 2001 and 2010, the prevalence of long-stay residents with HIV in NHs increased from 0.7% to 1.2%, a 71% increase. Long-stay residents with HIV were younger and less often female or white. For younger NH residents, rates of dementia were 20% and 16% for HIV+ and HIV- residents, respectively; they were 53% and 57%, respectively, for older residents. In adjusted analyses, younger HIV+ residents with dementia had greater odds of APS medication receipt than did HIV- residents (AOR = 1.3; 95% confidence interval [CI] = 1.2-1.4), but older HIV residents had lower odds (AOR = 0.9; 95% CI = 0.8-0.9). CONCLUSION:The prevalence of long-stay HIV+ NH residents has increased over time, and given the rapid aging of the HIV population, this increase is likely to have continued. This study raises concern about potential differential quality of care for (younger) residents with HIV in NHs, but not for those aged 65 years and older. These findings contribute to the evidence base needed to ensure high-quality care for younger and older HIV+ residents in NHs.
| S-EPMC6612569 | biostudies-literature
Project description:OBJECTIVES:Diagnoses extracted from physician notes are used to calculate hospital quality metrics; failure to document high-risk diagnoses may lead to the appearance of worse-than-expected outcomes for complex patients. Academic hospitals often rely on documentation authored by trainees, yet residents receive little training in this regard. In this study, we evaluate inpatient pediatric resident notes to determine which high-risk diagnoses are commonly missed and assess the efficacy of a multitiered intervention to improve the documentation of these diagnoses. METHODS:In a baseline review of 220 charts, 13 frequently missed high-risk diagnoses were identified in 2013. Interventions began in 2014, including physician education and reference cards. The intervention also included note template prompts for 4 of the diagnoses. Using a standardized rubric, we reviewed charts for 3 years (2013, 2014, and 2015). The average within-disease probability of missed high-risk diagnoses was compared across time. RESULTS:There was a decrease in the probability of undocumented target high-risk diagnoses after the intervention (52% vs 36% in 2014 [odds ratio = 0.51; P < .001] and 37% in 2015 [odds ratio = 0.50; P < .001]). Documentation of diagnoses prompted by the note template was not significantly better than those targeted by the other interventions alone (P = .55). CONCLUSIONS:Pediatric residents were significantly less likely to omit a high-risk diagnosis in their notes after implementation of our documentation improvement program, suggesting that curriculum development is an effective method of improving documentation, with the goal of improving the accuracy of health systems performance indices.
| S-EPMC7080527 | biostudies-literature
Project description:Introduction:We describe the development and implementation of tools medical educators or researchers can use for developing or analyzing residents' through attending physicians' clinical reasoning in an outpatient clinic setting. The resource includes two scenario-based simulations (i.e., diabetes, angina), implementation support materials, an open-ended postencounter form, and a think-aloud reflection protocol. Method:We designed two scenarios with potential case ambiguity and contextual factors to add complexity for studying clinical reasoning. The scenarios are designed to be used prior to an open-ended written exercise and a think-aloud reflection to elicit reasoning and reflection. We report on their implementation in a research context but developed them to be used in both educational and research settings. Results:Twelve physicians (five interns, three residents, and four attendings) considered between three and six differential diagnoses (M = 4.0) for the diabetes scenario and between three and nine differentials (M = 4.3) for angina. In think-aloud reflections, participants reconsidered their thinking between zero and 14 times (M = 3.5) for diabetes and zero and 11 times (M = 3.3) for angina. Cognitive load scores ranged from 4 to 8 (out of 10; M = 6.2) for diabetes and 5 to 8 (M = 6.6) for angina. Participants rated scenario authenticity between 4 and 5 (out of 5). Discussion:The potential case content ambiguity, along with the contextual factors (e.g., patient suggesting alternative diagnoses), provides a complex environment in which to explore or teach clinical reasoning.
| S-EPMC6346281 | biostudies-literature
Project description:US intelligence analysts must weigh up relevant evidence to assess the probability of their conclusions, and express this reasoning clearly in written reports for decision-makers. Typically, they work alone with no special analytic tools, and sometimes succumb to common probabilistic and causal reasoning errors. So, the US government funded a major research program (CREATE) for four large academic teams to develop new structured, collaborative, software-based methods that might achieve better results. Our team's method (BARD) is the first to combine two key techniques: constructing causal Bayesian network models (BNs) to represent analyst knowledge, and small-group collaboration via the Delphi technique. BARD also incorporates compressed, high-quality online training allowing novices to use it, and checklist-inspired report templates with a rudimentary AI tool for generating text explanations from analysts' BNs. In two prior experiments, our team showed BARD's BN-building assists probabilistic reasoning when used by individuals, with a large effect (Glass' ? 0.8) (Cruz et al., 2020), and even minimal Delphi-style interactions improve the BN structures individuals produce, with medium to very large effects (Glass' ? 0.5-1.3) (Bolger et al., 2020). This experiment is the critical test of BARD as an integrated system and possible alternative to business-as-usual for intelligence analysis. Participants were asked to solve three probabilistic reasoning problems spread over 5 weeks, developed by our team to test both quantitative accuracy and susceptibility to tempting qualitative fallacies. Our 256 participants were randomly assigned to form 25 teams of 6-9 using BARD and 58 individuals using Google Suite and (if desired) the best pen-and-paper techniques. For each problem, BARD outperformed this control with very large to huge effects (Glass' ? 1.4-2.2), greatly exceeding CREATE's initial target. We conclude that, for suitable problems, BARD already offers significant advantages over both business-as-usual and existing BN software. Our effect sizes also suggest BARD's BN-building and collaboration combined beneficially and cumulatively, although implementation differences decreased performances compared to Cruz et al. (2020), so interaction may have contributed. BARD has enormous potential for further development and testing of specific components and on more complex problems, and many potential applications beyond intelligence analysis.
| S-EPMC7314942 | biostudies-literature
Project description:Computer-based medical diagnostic decision support systems have been used for decades, initially as stand-alone applications. More recent versions have been tested for their effectiveness in enhancing the diagnostic ability of clinicians.To determine if viewing a rank-ordered list of diagnostic possibilities from a medical diagnostic decision support system improves residents' differential diagnoses or management plans.Twenty first-year internal medicine residents at Massachusetts General Hospital viewed 3 deidentified case descriptions of real patients. All residents completed a web-based questionnaire, entering the differential diagnosis and management plan before and after seeing the diagnostic decision support system's suggested list of diseases. In all 3 exercises, the actual case diagnosis was first on the system's list. Each resident served as his or her own control (pretest/posttest).For all 3 cases, a substantial percentage of residents changed their primary considered diagnosis after reviewing the system's suggested diagnoses, and a number of residents who had not initially listed a "further action" (laboratory test, imaging study, or referral) added or changed their management options after using the system. Many residents (20% to 65% depending on the case) improved their differential diagnosis from before to after viewing the system's suggestions. The average time to complete all 3 cases was 15.4 minutes. Most residents thought that viewing the medical diagnostic decision support system's list of suggestions was helpful.Viewing a rank-ordered list of diagnostic possibilities from a diagnostic decision support tool had a significant beneficial effect on the quality of first-year medicine residents' differential diagnoses and management plans.
| S-EPMC3399617 | biostudies-other