Dataset Information

Validity, reliability and feasibility of a new observation rating tool and a post encounter rating tool for the assessment of clinical reasoning skills of medical students during their internal medicine clerkship: a pilot study.

ABSTRACT:

Background

Systematic assessment of clinical reasoning skills of medical students in clinical practice is very difficult. This is partly caused by the lack of understanding of the fundamental mechanisms underlying the process of clinical reasoning.

Methods

We previously developed an observation tool to assess the clinical reasoning skills of medical students during clinical practice. This observation tool consists of an 11-item observation rating form (ORT). In the present study we verified the validity, reliability and feasibility of this tool and of an already existing post-encounter rating tool (PERT) in clinical practice among medical students during the internal medicine clerkship.

Results

Six raters each assessed the same 15 student-patient encounters. The internal consistency (Cronbach's alfa) for the (ORT) was 0.87 (0.71-0.84) and the 5-item (PERT) was 0.81 (0.71-0.87). The intraclass-correlation coefficient for single measurements was poor for both the ORT; 0.32 (p < 0.001) as well as the PERT; 0.36 (p < 0.001). The Generalizability study (G-study) and decision study (D-study) showed that 6 raters are required to achieve a G-coefficient of > 0.7 for the ORT and 7 raters for the PERT. The largest sources of variance are the interaction between raters and students. There was a consistent correlation between the ORT and PERT of 0.53 (p = 0.04).

Conclusions

The ORT and PERT are both feasible, valid and reliable instruments to assess students' clinical reasoning skills in clinical practice.

SUBMITTER: Haring CM

PROVIDER: S-EPMC7304120 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validity, reliability and feasibility of a new observation rating tool and a post encounter rating tool for the assessment of clinical reasoning skills of medical students during their internal medicine clerkship: a pilot study.

Haring Catharina M CM Klaarwater Claudia C R CCR Bouwmans Geert A GA Cools Bernadette M BM van Gurp Petra J M PJM van der Meer Jos W M JWM Postma Cornelis T CT

BMC medical education 20200619 1

<h4>Background</h4>Systematic assessment of clinical reasoning skills of medical students in clinical practice is very difficult. This is partly caused by the lack of understanding of the fundamental mechanisms underlying the process of clinical reasoning.<h4>Methods</h4>We previously developed an observation tool to assess the clinical reasoning skills of medical students during clinical practice. This observation tool consists of an 11-item observation rating form (ORT). In the present study w ...[more]

PMID: 32560648

Similar Datasets

Project description:BackgroundNon-technical skills (NTS) are essential for safe surgical practice as they impact workflow and patient outcomes. Observational tools to measure operating room (OR) teams' NTS have been introduced. However, there are none that account for the specific teamwork challenges introduced by robotic-assisted surgery (RAS). We set out to develop and content-validate a tool to assess multidisciplinary NTS in RAS.MethodologyStepwise, multi-method procedure. Observations in different surgical departments and a scoping literature review were first used to compile a set of RAS-specific teamwork behaviours. This list was refined and expert validated using a Delphi consensus approach consisting of qualitative interviews and a quantitative survey. Then, RAS-specific behaviours were merged with a well-established assessment tool on OR teamwork (NOTECHS II). Finally, the new tool-RAS-NOTECHS-was applied in standardized observations of real-world procedures to test its reliability (inter-rater agreement via intra-class correlations).ResultsOur scoping review revealed 5242 articles, of which 21 were included based on pre-established inclusion criteria. We elicited 16 RAS-specific behaviours from the literature base. These were synthesized with further 18 behavioural markers (obtained from 12 OR-observations) into a list of 26 behavioural markers. This list was reviewed by seven RAS experts and condensed to 15 expert-validated RAS-specific behavioural markers which were then merged into NOTECHS II. For five observations of urologic RAS procedures (duration: 13 h and 41 min), inter-rater agreement for identification of behavioural markers was strong. Agreement of RAS-NOTECHS scores indicated moderate to strong agreement.ConclusionsRAS-NOTECHS is the first observational tool for multidisciplinary NTS in RAS. In preliminary application, it has been shown to be reliable. Since RAS is rapidly increasing and challenges for effective and safe teamwork remain at the forefront of quality and safety of surgical care, RAS-NOTECHS may contribute to training and improvement efforts in technology-facilitated surgeries.

Project description:BackgroundRecent reports, including the Institute of Medicine's Improving Diagnosis in Health Care, highlight the pervasiveness and underappreciated harm of diagnostic error, and recommend enhancing health care professional education in diagnostic reasoning. However, little is known about clinical reasoning curricula at US medical schools.ObjectiveTo describe clinical reasoning curricula at US medical schools and to determine the attitudes of internal medicine clerkship directors toward teaching of clinical reasoning.DesignCross-sectional multicenter study.ParticipantsUS institutional members of the Clerkship Directors in Internal Medicine (CDIM).Main measuresExamined responses to a survey that was emailed in May 2015 to CDIM institutional representatives, who reported on their medical school's clinical reasoning curriculum.Key resultsThe response rate was 74% (91/123). Most respondents reported that a structured curriculum in clinical reasoning should be taught in all phases of medical education, including the preclinical years (64/85; 75%), clinical clerkships (76/87; 87%), and the fourth year (75/88; 85%), and that more curricular time should be devoted to the topic. Respondents indicated that most students enter the clerkship with only poor (25/85; 29%) to fair (47/85; 55%) knowledge of key clinical reasoning concepts. Most institutions (52/91; 57%) surveyed lacked sessions dedicated to these topics. Lack of curricular time (59/67, 88%) and faculty expertise in teaching these concepts (53/76, 69%) were identified as barriers.ConclusionsInternal medicine clerkship directors believe that clinical reasoning should be taught throughout the 4 years of medical school, with the greatest emphasis in the clinical years. However, only a minority reported having teaching sessions devoted to clinical reasoning, citing a lack of curricular time and faculty expertise as the largest barriers. Our findings suggest that additional institutional and national resources should be dedicated to developing clinical reasoning curricula to improve diagnostic accuracy and reduce diagnostic error.

Project description:IntroductionAssessment of medical students' clinical skills (CS) remains a challenge. Little is known about early predictors of future CS performance. This study examines the relationship between students' pre-clerkship clinical skills (PCCS) performance and year 3 clerkship performance measures.MethodsThe authors performed a retrospective analysis of four medical student cohorts who matriculated between 2014 and 2017 and participated in a longitudinal pre-clerkship CS curriculum. A total of 440 students were included in the analyses. Students' clinical skills were assessed through a series of PCCS exams, each consisting of a single standardized patient encounter. First-year PCCS exams assessed history taking, physical examination, professionalism, and communication skills; second-year PCCS exams also assessed clinical documentation and clinical reasoning skills. Evaluators assigned a grade of "satisfactory," "borderline," or "unsatisfactory" for each skill set. Regression analyses compared year 3 performance outcomes between students with one or more "unsatisfactory" or "borderline" PCCS skill set grades and students assessed as "satisfactory" for all PCCS skill set assessments.ResultsThirty-two percent (n = 140) of the 440 students had at least one borderline or unsatisfactory (US) PCCS skill set grade. These students performed significantly worse on year 3 National Board of Medical Examiner subject exams, workplace-based clinical performance evaluations, and overall year 3 performance compared to students who passed all PCCS exam components. In addition, a higher percentage of students with PCCS performance deficiencies failed the United States Medical Licensing Examination Step 2 CS exam on the first attempt versus students who passed all PCCS exam components.ConclusionsPCCS exam performance at our institution aligned with future student performance on multiple year 3 clerkship outcome measures. This pre-clerkship performance data can be used to identify at-risk students who would benefit from additional resources to achieve competency in the clerkship environment and future medical training.Supplementary informationThe online version contains supplementary material available at 10.1007/s40670-022-01519-8.

Dataset Information

Validity, reliability and feasibility of a new observation rating tool and a post encounter rating tool for the assessment of clinical reasoning skills of medical students during their internal medicine clerkship: a pilot study.

Background

Methods

Results

Conclusions

Publications

Validity, reliability and feasibility of a new observation rating tool and a post encounter rating tool for the assessment of clinical reasoning skills of medical students during their internal medicine clerkship: a pilot study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets