Dataset Information

Application of an objective structured clinical examination to evaluate and monitor intern's proficiency of hand hygiene and personal protective equipment use in the United States.

ABSTRACT: PURPOSE:To determine if an objective structured clinical examination (OSCE) could be used to evaluate and monitor hand hygiene and personal protective equipment (PPE) proficiency for medical interns in the United States. METHODS:Interns in July 2015 (N=123, Cohort 1) without OSCE-based contact precaution evaluation and teaching were evaluated early 2016 by OSCE for hand hygiene and PPE proficiency. They performed poorly. Therefore, the new interns entering July 2016 (N=151, Cohort 2) were immediately tested in the same OSCE station as Cohort 1 and provided feedback and teaching. Cohort 2 was then retested in the OSCE station early 2017. The Mann Whitney U test was used to compare Cohort 1 vs. Cohort 2 performances on checklist items. Cohort 2 performance differences at the beginning and end of the intern year were compared using McNemar's X2 test for paired nominal data. RESULTS:Checklist items were scored, summed and reported as percent correct. In Cohort 2, the mean percent correct was higher in posttest than pretest, 92% vs. 77% )(P <0 .0001). The passing rate (100% correct) was significantly higher, 55% vs. 16%. Comparing Cohort 1 and Cohort 2 at the end of intern year, the mean percent correct was higher for Cohort 2 compared to Cohort 1, 95% vs 90% (P < 0.0001). 55% of the Cohort 2 passed (a perfect score) compared to 24% in Cohort 1 (P < 0.0001). CONCLUSION:An OSCE can be utilized to evaluate and monitor hand hygiene and PPE proficiency for interns in the United States.

SUBMITTER: Nagoshi Y

PROVIDER: S-EPMC6819956 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Application of an objective structured clinical examination to evaluate and monitor intern's proficiency of hand hygiene and personal protective equipment use in the United States.

Nagoshi Ying Y Cooper Lou Ann LA Meyer Lynne L Cherabuddi Kartik K Close Julia J Dow Jamie J Markham Merry Jennifer MJ Stalvey Carolyn C

Journal of educational evaluation for health professions 20191015

<h4>Purpose</h4>To determine if an objective structured clinical examination (OSCE) could be used to evaluate and monitor hand hygiene and personal protective equipment (PPE) proficiency for medical interns in the United States.<h4>Methods</h4>Interns in July 2015 (N=123, Cohort 1) without OSCE-based contact precaution evaluation and teaching were evaluated early 2016 by OSCE for hand hygiene and PPE proficiency. They performed poorly. Therefore, the new interns entering July 2016 (N=151, Cohort ...[more]

PMID: 31614408

Similar Datasets

Project description:Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE (MS-OSCE) where all students had to manage the same chief complaint at a station but it's underlying scenarios being randomly changed during students' rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios. Methods: We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on students' grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model. Results: A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach's alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students' skills. Conclusions: The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students.

Project description:BACKGROUND:Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students' performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE:This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students' radiology knowledge and the reproducibility of this assessment tool. METHODS:We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students' clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS:A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (-0.02; P=.48), adjusting for the medical school year. CONCLUSIONS:Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.

Project description:OBJECTIVES:Sources of bias, such as the examiners, domains and stations, can influence the student marks in objective structured clinical examination (OSCE). This study describes the extent to which the facets modelled in an OSCE can contribute to scoring variance and how they fit into a Many-Facet Rasch Model (MFRM) of OSCE performance. A further objective is to identify the functioning of the rating scale used. DESIGN:A non-experimental cross-sectional design. PARTICIPANTS AND SETTINGS:An MFRM was used to identify sources of error (eg, examiner, domain and station), which may influence the student outcome. A 16-station OSCE was conducted for 329 final year medical students. Domain-based marking was applied, each station using a sample from eight defined domains across the whole OSCE. The domains were defined as follows: communication skills, professionalism, information gathering, information giving, clinical interpretation, procedure, diagnosis and management. The domains in each station were weighted to ensure proper attention to the construct of the individual station. Four facets were assessed: students, examiners, domains and stations. RESULTS:The results suggest that the OSCE data fit the model, confirming that an MFRM approach was appropriate to use. The variable map allows a comparison with and between the facets of students, examiners, domains and stations and the 5-point score for each domain with each station as they are calibrated to the same scale. Fit statistics showed that the domains map well to the performance of the examiners. No statistically significant difference between examiner sensitivity (3.85 logits) was found. However, the results did suggest examiners were lenient and that some behaved inconsistently. The results also suggest that the functioning of response categories on the 5-point rating scale need further examination and optimisation. CONCLUSIONS:The results of the study have important implications for examiner monitoring and training activities, to aid assessment improvement.

Dataset Information

Application of an objective structured clinical examination to evaluate and monitor intern's proficiency of hand hygiene and personal protective equipment use in the United States.

Publications

Application of an objective structured clinical examination to evaluate and monitor intern's proficiency of hand hygiene and personal protective equipment use in the United States.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets