Dataset Information

Using Virtual Patients to Explore the Clinical Reasoning Skills of Medical Students: Mixed Methods Study.

ABSTRACT:

Background

Improving clinical reasoning skills-the thought processes used by clinicians to formulate appropriate questions and diagnoses-is essential for reducing missed diagnostic opportunities. The electronic Clinical Reasoning Educational Simulation Tool (eCREST) was developed to improve the clinical reasoning of future physicians. A feasibility trial demonstrated acceptability and potential impacts; however, the processes by which students gathered data were unknown.

Objective

This study aims to identify the data gathering patterns of final year medical students while using eCREST and how eCREST influences the patterns.

Methods

A mixed methods design was used. A trial of eCREST across 3 UK medical schools (N=148) measured the potential effects of eCREST on data gathering. A qualitative think-aloud and semistructured interview study with 16 medical students from one medical school identified 3 data gathering strategies: Thorough, Focused, and Succinct. Some had no strategy. Reanalysis of the trial data identified the prevalence of data gathering patterns and compared patterns between the intervention and control groups. Patterns were identified based on 2 variables that were measured in a patient case 1 month after the intervention: the proportion of Essential information students identified and the proportion of irrelevant information gathered (Relevant). Those who scored in the top 3 quartiles for Essential but in the lowest quartile for Relevant displayed a Thorough pattern. Those who scored in the top 3 quartiles for Relevant but in the lowest quartile for Essential displayed a Succinct pattern. Those who scored in the top 3 quartiles on both variables displayed a Focused pattern. Those whose scores were in the lowest quartile on both variables displayed a Nonspecific pattern.

Results

The trial results indicated that students in the intervention group were more thorough than those in the control groups when gathering data. The qualitative data identified data gathering strategies and the mechanisms by which eCREST influenced data gathering. Students reported that eCREST promoted thoroughness by prompting them to continuously reflect and allowing them to practice managing uncertainty. However, some found eCREST to be less useful, and they randomly gathered information. Reanalysis of the trial data revealed that the intervention group was significantly more likely to display a Thorough data gathering pattern than controls (21/78, 27% vs 6/70, 9%) and less likely to display a Succinct pattern (13/78, 17% vs 20/70, 29%; χ²₃=9.9; P=.02). Other patterns were similar across groups.

Conclusions

Qualitative data suggested that students applied a range of data gathering strategies while using eCREST and that eCREST encouraged thoroughness by continuously prompting the students to reflect and manage their uncertainty. Trial data suggested that eCREST led students to demonstrate more Thorough data gathering patterns. Virtual patients that encourage thoroughness could help future physicians avoid missed diagnostic opportunities and enhance the delivery of clinical reasoning teaching.

SUBMITTER: Plackett R

PROVIDER: S-EPMC8214179 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectivesTo evaluate the impact and transferability of a novel teaching method on virtual communication skills for final year medical students.DesignMixed-methods, interventional before-and-after study.SettingNHS Lanarkshire, Scotland.Participants21 final year medical students on their obstetrics and gynaecology (O&G) placement from September to December 2020.InterventionsA two-part teaching session on virtual communication skills.Main outcome measuresSelf-reported confidence in conducting consultations preteaching and post-teaching, exposure to virtual consultations, usefulness of teaching and transferability to primary care. Data were collected using preteaching and post-teaching evaluation tools and an online survey.ResultsOf 21 participants, 1 student did not attend the second session so was excluded from post-teaching evaluation results and the online survey. Preteaching results were collected from 21 participants and post-teaching results from 20. Mean confidence scores increased across all domains post-teaching. Mean confidence in opening the consultation increased from 2.67 (95% CI 2.21 to 3.13) to 4.70 (95% CI 4.50 to 4.90); history-taking from 3.38 (95% CI 3.07 to 3.69) to 4.45 (95% CI 4.19 to 4.71); decision-making and forming a management plan from 2.62 (95% CI 2.28 to 2.96) to 3.90 (95% CI 3.66 to 4.14) and closing the consultation from 2.81 (95% CI 2.45 to 3.17) to 4.60 (95% CI 4.38 to 4.81). There was no change in exposure to virtual consultations during O&G placement. 16 (80%) participants responded to the online survey; 14 (87.5%) rated the sessions 'very useful' and all 16 considered them worthwhile continuing. 12 (75%) had the opportunity to practise virtual consultations on general practitioner, mostly via telephone.ConclusionsWe found that teaching students virtual consultation skills improved short term confidence and were transferable to primary care placements. Future research is suggested to assess this teaching model following adaptation and incorporation into medical education and training across specialties and grades. It would be useful to evaluate the impact on competence post intervention through observed skills.

Project description:BackgroundTraining for the fundus examination using traditional teaching is challenging, resulting in low generalist physicians' confidence in performing the funduscopic examination. There is growing evidence suggesting a flexible e-learning video approach's value in teaching physical examination procedures. However, whether the flexible e-learning video approach is superior to the traditional, face-to-face (F2F) lecture-based teaching for the funduscopic exam and the cognitive processes supporting its effectiveness has not yet been determined.MethodsWe conducted a sequential explanatory mixed-method study to compare the flexible e-learning video approach's effectiveness versus the F2F lecture-based approach for teaching the funduscopic exam to medical students at Chiba University in Japan. Medical students were randomly assigned to either a flexible e-learning video approach group or a F2F lecture approach group. We then quantitatively measured the diagnostic accuracy of funduscopic findings before and after attending the specific classrooms. Next, we conducted student focus groups to explore the students' thinking processes in the flexible e-learning video approach vs. the F2F lecture-based teaching of fundus examination. The qualitative data were analyzed using the qualitative content analysis method.ResultsThe mean diagnostic accuracy scores in the post-test significantly increased from pre-test in the intervention group (36.6 to 63.4%, p < 0.001). Post-post comparisons across the two groups revealed a significant difference (intervention group 63.4% vs. control group 34.6%, p < 0.001). Six semi-structured focused group interviews were conducted (n = 36). In the flexible e-learning video approach group, we identified ten categories corresponding to four levels of the revised Bloom's taxonomy: remember, understand, apply, analyze. Five categories were identified in the traditional F2F lecture approach group corresponding to three revised Bloom's taxonomy levels: understand, apply, analyze. Interrater reliability was substantial (Cohen's kappa = 0.81).ConclusionsTeaching medical students funduscopic examination using the flexible e-learning video approach leads to improved diagnostic accuracy of funduscopic examinations. The flexible e-learning video teaching method enabled higher cognitive activity levels than the traditional, lecture-based classroom, as assessed using the revised Bloom's taxonomy.Trial registrationThis study was registered with the University Hospital Medical Information Network Clinical Trials Registry on 08/02/2020 (Unique trial number: UMIN 000039434 ).

Project description:BackgroundClinical reasoning plays a major role in the ability of doctors to make a diagnosis and reach treatment decisions. This paper describes the use of four clinical reasoning tests in the second National Medical Science Olympiad in Iran: key features (KF), script concordance (SCT), clinical reasoning problems (CRP) and comprehensive integrative puzzles (CIP). The purpose of the study was to design a multi instrument for multiple roles approach in clinical reasoning field based on the theoretical framework, KF was used to measure data gathering, CRP was used to measure hypothesis formation, SCT and CIP were used to measure hypothesis evaluation and investigating the combined use of these tests in the Olympiad. A bank of clinical reasoning test items was developed for emergency medicine by a scientific expert committee representing all the medical schools in the country. These items were pretested by a reference group and the results were analyzed to select items that could be omitted. Then 135 top-ranked medical students from 45 medical universities in Iran participated in the clinical domain of the Olympiad. The reliability of each test was calculated by Cronbach's alpha. Item difficulty and the correlation between each item and the total score were measured. The correlation between the students' final grade and each of the clinical reasoning tests was calculated, as was the correlation between final grades and another measure of knowledge, i.e., the students' grade point average.ResultsThe combined reliability for all four clinical reasoning tests was 0.91. Of the four clinical reasoning tests we compared, reliability was highest for CIP (0.91). The reliability was 0.83 for KF, 0.78 for SCT and 0.71 for CRP. Most of the tests had an acceptable item difficulty level between 0.2 and 0.8. The correlation between the score for each item and the total test score for each of the four tests was positive. The correlations between scores for each test and total score were highest for KF and CIP. The correlation between scores for each test and grade point average was low to intermediate for all four of the tests.ConclusionThe combination of these four clinical reasoning tests is a reliable evaluation tool that can be implemented to assess clinical reasoning skills in talented undergraduate medical students, however these data may not generalizable to whole medical students population. The CIP and KF tests showed the greatest potential to measure clinical reasoning skills. Grade point averages did not necessarily predict performance in the clinical domain of the national competitive examination for medical school students.

Project description:PurposeDeveloping medical students' clinical reasoning requires a structured longitudinal curriculum with frequent targeted assessment and feedback. Performance-based assessments, which have the strongest validity evidence, are currently not feasible for this purpose because they are time-intensive to score. This study explored the potential of using machine learning technologies to score one such assessment-the diagnostic justification essay.MethodFrom May to September 2018, machine scoring algorithms were trained to score a sample of 700 diagnostic justification essays written by 414 third-year medical students from the Southern Illinois University School of Medicine classes of 2012-2017. The algorithms applied semantically based natural language processing metrics (e.g., coherence, readability) to assess essay quality on 4 criteria (differential diagnosis, recognition and use of findings, workup, and thought process); the scores for these criteria were summed to create overall scores. Three sources of validity evidence (response process, internal structure, and association with other variables) were examined.ResultsMachine scores correlated more strongly with faculty ratings than faculty ratings did with each other (machine: .28-.53, faculty: .13-.33) and were less case-specific. Machine scores and faculty ratings were similarly correlated with medical knowledge, clinical cognition, and prior diagnostic justification. Machine scores were more strongly associated with clinical communication than were faculty ratings (.43 vs .31).ConclusionsMachine learning technologies may be useful for assessing medical students' long-form written clinical reasoning. Semantically based machine scoring may capture the communicative aspects of clinical reasoning better than faculty ratings, offering the potential for automated assessment that generalizes to the workplace. These results underscore the potential of machine scoring to capture an aspect of clinical reasoning performance that is difficult to assess with traditional analytic scoring methods. Additional research should investigate machine scoring generalizability and examine its acceptability to trainees and educators.

Project description:ObjectivesThis study investigates the effectiveness of a virtual format of an advanced communication skills observed structured clinical examination (OSCE) for senior medical students in comparison to an in-person format. The study also examines the emotional support students experience in the virtual setting. Our analysis was based on quantitative data collected through objective checklists and post-OSCE survey results.MethodsThe virtual OSCE was a revision of an earlier in-person formative advanced communication skills OSCE for fourth-year medical students. Student performances were assessed by self and peers using objective checklists-the modified Master Interview Rating Scale (mMIRS) and Communication Behavior Checklist (CBC). The mMIRS measured interview process such as avoiding jargon and demonstrating empathy. The CBC examined interview content which included tasks specific to the content of the case. The OSCE was followed by a faculty-led debrief and quantitative survey. The virtual OSCE was conducted in 2021, and the results of the checklists and survey were compared with those collected from two earlier in-person OSCEs.ResultsEighty-three students participated in the virtual OSCE. There was no difference in mMIRS scores between the virtual and in-person OSCE. Overall CBC scores were lower in the virtual OSCE compared to in-person (p < 0.05). Sixty-seven out of 83 (80.7%) students completed the post-OSCE survey. There were no differences between the virtual and in-person OSCE in terms of educational value, whether the OSCE would change the way participants talk to patients, and preparedness to have serious conversations with patients. All respondents somewhat or strongly agreed with feeling emotionally supported during the virtual OSCE.ConclusionThe virtual format was a suitable alternative to an in-person, formative, advanced communication skills OSCE for medical students. The virtual OSCE was educationally effective and was met with student satisfaction and a sense of emotional support. Future virtual iterations must ensure adequate instruction on interview content.

Project description:OBJECTIVE:Fostering clinical reasoning is a mainstay of medical education. Based on the clinicopathological conferences, we propose a case-based peer teaching approach called clinical case discussions (CCDs) to promote the respective skills in medical students. This study compares the effectiveness of different CCD formats with varying degrees of social interaction in fostering clinical reasoning. DESIGN, SETTING, PARTICIPANTS:A single-centre randomised controlled trial with a parallel design was conducted at a German university. Study participants (N=106) were stratified and tested regarding their clinical reasoning skills right after CCD participation and 2 weeks later. INTERVENTION:Participants worked within a live discussion group (Live-CCD), a group watching recordings of the live discussions (Video-CCD) or a group working with printed cases (Paper-Cases). The presentation of case information followed an admission-discussion-summary sequence. PRIMARY AND SECONDARY OUTCOME MEASURES:Clinical reasoning skills were measured with a knowledge application test addressing the students' conceptual, strategic and conditional knowledge. Additionally, subjective learning outcomes were assessed. RESULTS:With respect to learning outcomes, the Live-CCD group displayed the best results, followed by Video-CCD and Paper-Cases, F(2,87)=27.07, p<0.001, partial η2=0.384. No difference was found between Live-CCD and Video-CCD groups in the delayed post-test; however, both outperformed the Paper-Cases group, F(2,87)=30.91, p<0.001, partial η2=0.415. Regarding subjective learning outcomes, the Live-CCD received significantly better ratings than the other formats, F(2,85)=13.16, p<0.001, partial η2=0.236. CONCLUSIONS:This study demonstrates that the CCD approach is an effective and sustainable clinical reasoning teaching resource for medical students. Subjective learning outcomes underline the importance of learner (inter)activity in the acquisition of clinical reasoning skills in the context of case-based learning. Higher efficacy of more interactive formats can be attributed to positive effects of collaborative learning. Future research should investigate how the Live-CCD format can further be improved and how video-based CCDs can be enhanced through instructional support.