Dataset Information

Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students' Ability to Estimate Probability of a Diagnosis: A Randomized Clinical Trial.

ABSTRACT:

Importance

Clinicians use probability estimates to make a diagnosis. Teaching students to make more accurate probability estimates could improve the diagnostic process and, ultimately, the quality of medical care.

Objective

To test whether novice clinicians can be taught to make more accurate bayesian revisions of diagnostic probabilities using teaching methods that apply either explicit conceptual instruction or repeated examples.

Design, setting, and participants

A randomized clinical trial of 2 methods for teaching bayesian updating and diagnostic reasoning was performed. A web-based platform was used for consent, randomization, intervention, and testing of the effect of the intervention. Participants included 61 medical students at McMaster University and Eastern Virginia Medical School recruited from May 1 to September 30, 2018.

Interventions

Students were randomized to (1) receive explicit conceptual instruction regarding diagnostic testing and bayesian revision (concept group), (2) exposure to repeated examples of cases with feedback regarding posttest probability (experience group), or (3) a control condition with no conceptual instruction or repeated examples.

Main outcomes and measures

Students in all 3 groups were tested on their ability to update the probability of a diagnosis based on either negative or positive test results. Their probability revisions were compared with posttest probability revisions that were calculated using the Bayes rule and known test sensitivity and specificity.

Results

Of the 61 participants, 22 were assigned to the concept group, 20 to the experience group, and 19 to the control group. Approximate age was 25 years. Two participants were first-year; 37, second-year; 12, third-year; and 10, fourth-year students. Mean (SE) probability estimates of students in the concept group were statistically significantly closer to calculated bayesian probability than the other 2 groups (concept, 0.4%; [0.7%]; experience, 3.5% [0.7%]; control, 4.3% [0.7%]; P < .001). Although statistically significant, the differences between groups were relatively modest, and students in all groups performed better than expected, based on prior reports in the literature.

Conclusions and relevance

The study showed a modest advantage for students who received theoretical instruction on bayesian concepts. All participants' probability estimates were, on average, close to the bayesian calculation. These findings have implications for how to teach diagnostic reasoning to novice clinicians.

Trial registration

ClinicalTrials.gov identifier: NCT04130607.

SUBMITTER: Brush JE

PROVIDER: S-EPMC7027434 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Within computational neuroscience, the algorithmic and neural basis of structure learning remains poorly understood. Concept learning is one primary example, which requires both a type of internal model expansion process (adding novel hidden states that explain new observations), and a model reduction process (merging different states into one underlying cause and thus reducing model complexity via meta-learning). Although various algorithmic models of concept learning have been proposed within machine learning and cognitive science, many are limited to various degrees by an inability to generalize, the need for very large amounts of training data, and/or insufficiently established biological plausibility. Using concept learning as an example case, we introduce a novel approach for modeling structure learning-and specifically state-space expansion and reduction-within the active inference framework and its accompanying neural process theory. Our aim is to demonstrate its potential to facilitate a novel line of active inference research in this area. The approach we lay out is based on the idea that a generative model can be equipped with extra (hidden state or cause) "slots" that can be engaged when an agent learns about novel concepts. This can be combined with a Bayesian model reduction process, in which any concept learning-associated with these slots-can be reset in favor of a simpler model with higher model evidence. We use simulations to illustrate this model's ability to add new concepts to its state space (with relatively few observations) and increase the granularity of the concepts it currently possesses. We also simulate the predicted neural basis of these processes. We further show that it can accomplish a simple form of "one-shot" generalization to new stimuli. Although deliberately simple, these simulation results highlight ways in which active inference could offer useful resources in developing neurocomputational models of structure learning. They provide a template for how future active inference research could apply this approach to real-world structure learning problems and assess the added utility it may offer.

Project description:Physics is fundamental to secure future needs for scientific and technological competence (Angell et al., 2004), but many countries experience a drop in students' performances in international assessments (Organisation for Economic Co-operation Development [OECD], 2018), as well as in rates of enrolment in undergraduate programs in scientific disciplines (STEM). Socio-constructivist theories have produced a reforming movement in several educational systems, in particular in the area of sciences, but teacher often consider them an idealistic view of education and do not consider themselves metacognitively competent enough to foster thinking in the classroom. In this study, we investigated the efficacy of different teaching methods on high-school students' conceptual knowledge of physics, after the effect of science-related beliefs and critical thinking skills was controlled. We adopted a mixed-method with sequential design, in which quantitative and qualitative data flow are inter-mixed. In specific, we interviewed four high school physics teachers to identify teaching approaches (qualitative approach) and compared them in terms of efficacy on students' performances (quantitative approach). Four teachers and 77 10th grade students participated. Teachers were interviewed during the school years and asked questions about their teaching experience, their teaching approach (Kang and Wallace, 2005) and their epistemic beliefs (Tsai, 2002). Students performances in Science-related beliefs (Conley et al., 2004), critical thinking (Cornell Critical Thinking Test Level X, Millman et al., 2005), and conceptual knowledge in physics (The Force and Motion Conceptual Evaluation, Ramlo, 2002) were evaluated twice, at the beginning and at the end of the school year. The independent-sample t-tests on pre-test variables did not reveal any statistically significant difference between groups. Results from the complex samples GLM revealed statistically significant differences on post-test scores in conceptual knowledge in physics, after the effect of covariates was controlled. Overall, the study contributes to our understanding on current teaching practices in school, and their effect on students' conceptual understanding of physics concepts.