Dataset Information

Objective structured clinical examinations (OSCE) performance among Quebec urology residents: A retrospective study from 2008-2019.

ABSTRACT: INTRODUCTION:We aimed to compare objective structured clinical examinations (OSCE) performance of residents from four Canadian urology programs, based on resident and station characteristics. We also aimed to evaluate OSCE contents by station type and subspecialty. METHODS:Scores of 109 postgraduate year (PGY)-3 to PGY-5 residents were retrospectively reviewed from 19 OSCEs from May 2008 to February 2019. Scores were grouped by station type/subspecialty, PGY level, medical graduate type (Canadian medical graduate [CMG], international medical graduate [IMG]), sex, and choice of fellowship/practice. Linear mixed modelling was performed to obtain least square means to account for repeated measures. RESULTS:Score increases from PGY-3 to PGY-5 were significant for all station types and subspecialties (p?0.001). Scores were similar between male and female residents, and between CMGs and IMGs, except in visual recognition examinations (VREs) (males: 44.3±1.0, females: 39.0±1.6, p=0.005; IMG: 47.3±1.7, CMG: 41.6±0.9, p=0.004). Relative to uro-oncology stations, scores were lower in andrology (p=0.010) and functional urology (p<0.001). More female residents chose pediatric (14.3% vs. 1.5%, p=0.024) and functional urology fellowships (17.9% vs. 2.9%, p=0.021). More male residents chose endourology/robotic fellowships (30.9% vs. 10.7%, p=0.042). No associations between subspecialty scores and choice of fellowship/practice were found. Oral stations and VREs were more frequent than telephone stations. Uro-oncology and pediatric urology were more frequent than other subspecialties. CONCLUSIONS:Scores improved with higher PGY level. IMGs and male residents scored better in VREs. Scores were lower in functional urology. There was no correlation between subspecialty score and choice of fellowship/practice. Subspecialties and forms of evaluation were not equally represented.

SUBMITTER: Garabed LR

PROVIDER: S-EPMC7492032 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Objective structured clinical examinations (OSCE) performance among Quebec urology residents: A retrospective study from 2008-2019.

Garabed Laurianne Rita LR Almarzouq Ahmad A Hu Jason J Andonian Sero S El-Sherbiny Mohamed M Fahmy Nader N

Canadian Urological Association journal = Journal de l'Association des urologues du Canada 20200901 9

<h4>Introduction</h4>We aimed to compare objective structured clinical examinations (OSCE) performance of residents from four Canadian urology programs, based on resident and station characteristics. We also aimed to evaluate OSCE contents by station type and subspecialty.<h4>Methods</h4>Scores of 109 postgraduate year (PGY)-3 to PGY-5 residents were retrospectively reviewed from 19 OSCEs from May 2008 to February 2019. Scores were grouped by station type/subspecialty, PGY level, medical graduat ...[more]

PMID: 32223876

Similar Datasets

Project description:BackgroundTeaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales.MethodsWe conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues.ResultsData of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate.DiscussionOur results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically sound results of the OSCEs assessing communication skills. This is especially important given that most OSCE rating scales are used for summative assessment, and thus have an impact on medical students' academic success.

Project description:This study aimed to investigate the impact of student e-learning on the development of clinical competencies. The study participants were 3rd year students (n = 43) at a private mid-sized medical school located in a South Korean suburb on a four-year medical program. Educational intervention was implemented to enhance student clinical performance. Students engaged in learning activities that intended to promote their self-directed learning abilities and clinical performances using e-learning resources. Intervention was conducted for the duration of six months during the 3rd year and its effectiveness was investigated by comparing student performances in OSCEs in a pre- and post- comparison format and also by comparing them with national scores. In addition, student perceptions of the impact of e-learning on their OSCE performances were assessed using a questionnaire, which included 36 items that elicited student perceptions of their experiences of e-learning and readiness for e-learning. Student OSCE scores improved significantly after educational intervention in all domains of clinical competencies assessed and for total scores (p < 0.001). Furthermore, students achieved higher OSCE scores than national average scores in the post-test, whereas they had performed lower than national average scores in the pre-test. Students showed neutral or slightly positive responses to the effectiveness of e-learning, and their perceptions of e-learning were not associated with their e-learning readiness scores. The study shows student OSCE performance improved significantly after educational intervention, which indicate the effectiveness of e-learning to support student learning of clinical performance. Despite significant improvements in student OSCE scores after e-learning, their perceptions of its effectiveness were neutral. Furthermore, student perceptions of e-learning were not associated with their readiness for it. Suggestions are made to help students use e-learning more effectively to enhance their clinical competencies.

Project description:PurposeEnsuring that examiners in different parallel circuits of objective structured clinical examinations (OSCEs) judge to the same standard is critical to the chain of validity. Recent work suggests examiner-cohort (i.e., the particular group of examiners) could significantly alter outcomes for some candidates. Despite this, examiner-cohort effects are rarely examined since fully nested data (i.e., no crossover between the students judged by different examiner groups) limit comparisons. In this study, the authors aim to replicate and further develop a novel method called Video-based Examiner Score Comparison and Adjustment (VESCA), so it can be used to enhance quality assurance of distributed or national OSCEs.MethodIn 2019, 6 volunteer students were filmed on 12 stations in a summative OSCE. In addition to examining live student performances, examiners from 8 separate examiner-cohorts scored the pool of video performances. Examiners scored videos specific to their station. Video scores linked otherwise fully nested data, enabling comparisons by Many Facet Rasch Modeling. Authors compared and adjusted for examiner-cohort effects. They also compared examiners' scores when videos were embedded (interspersed between live students during the OSCE) or judged later via the Internet.ResultsHaving accounted for differences in students' ability, different examiner-cohort scores for the same ability of student ranged from 18.57 of 27 (68.8%) to 20.49 (75.9%), Cohen's d = 1.3. Score adjustment changed the pass/fail classification for up to 16% of students depending on the modeled cut score. Internet and embedded video scoring showed no difference in mean scores or variability. Examiners' accuracy did not deteriorate over the 3-week Internet scoring period.ConclusionsExaminer-cohorts produced a replicable, significant influence on OSCE scores that was unaccounted for by typical assessment psychometrics. VESCA offers a promising means to enhance validity and fairness in distributed OSCEs or national exams. Internet-based scoring may enhance VESCA's feasibility.

Dataset Information

Objective structured clinical examinations (OSCE) performance among Quebec urology residents: A retrospective study from 2008-2019.

Publications

Objective structured clinical examinations (OSCE) performance among Quebec urology residents: A retrospective study from 2008-2019.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets