Dataset Information

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

ABSTRACT:

Unlabelled

Background?From 2010 to 2011, more than 70% of the clinical rotation competency evaluations for nephrology fellows in our program were rated "superior" using a 9-point Likert scale, suggesting some degree of "grade inflation." Objective?We sought to assess the efficacy of a 5-point centered rotation evaluation in reducing grade inflation. Methods?This retrospective cohort study of the impact of faculty education and a 5-point rotation evaluation on grade inflation was measured by superior item rating frequency and proportion of evaluations without superior ratings. The 5-point evaluation centered performance at the level expected for stage of training. Faculty education began in 2011-2012. The 5-point centered evaluation was introduced in 2012-2013 and used exclusively thereafter. A total of 68 evaluations, using the 9-point Likert scale, and 63 evaluations, using the 5-point centered scale, were performed after first-year fellow clinical rotations. Nine to 12 faculty members participated yearly. Results?Faculty education alone was associated with fewer superior ratings from 2010-2011 to 2011-2012 (70.5% versus 48.3%, P?=?.001), declining further with 5-point centered scale introduction (2012-2013; 48.3% versus 35.6%; P?=?.012). Superior ratings declined with 5-point centered versus 9-point Likert scales (37.3% versus 59.3%, P?=?.001), specifically for medical knowledge, patient care, practice-based learning and improvement, and professionalism. On logistic regression, evaluations without superior scores were more likely for 5-point centered versus 9-point Likert scales (adjusted odds ratio [aOR]?=?8.26; 95% CI 1.53-44.64; P?=?.014) and associated with faculty identifier (aOR= 1.18; 95% CI 1.03-1.35; P?=?.013), but not fellow identifier or training year quarter.

Conclusions

Grade inflation was reduced with faculty education and the 5-point centered evaluation scale.

SUBMITTER: Yuan CM

PROVIDER: S-EPMC4857519 | biostudies-literature | 2016 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Yuan Christina M CM Nee Robert R Abbott Kevin C KC Oliver James D JD

Journal of graduate medical education 20160501 2

<h4>Unlabelled</h4>Background From 2010 to 2011, more than 70% of the clinical rotation competency evaluations for nephrology fellows in our program were rated "superior" using a 9-point Likert scale, suggesting some degree of "grade inflation." Objective We sought to assess the efficacy of a 5-point centered rotation evaluation in reducing grade inflation. Methods This retrospective cohort study of the impact of faculty education and a 5-point rotation evaluation on grade inflation was measured ...[more]

PMID: 27168886

Similar Datasets

Project description:ImportanceAlthough implicit bias in medical training has long been suspected, it has been difficult to study using objective measures, and the influence of sex and gender in the evaluation of medical trainees is unknown. The emergency medicine (EM) milestones provide a standardized framework for longitudinal resident assessment, allowing for analysis of resident performance across all years and programs at a scope and level of detail never previously possible.ObjectiveTo compare faculty-observed training milestone attainment of male vs female residency training.Design, setting, and participantsThis multicenter, longitudinal, retrospective cohort study took place at 8 community and academic EM training programs across the United States from July 1, 2013, to July 1, 2015, using a real-time, mobile-based, direct-observation evaluation tool. The study examined 33 456 direct-observation subcompetency evaluations of 359 EM residents by 285 faculty members.Main outcomes and measuresMilestone attainment for male and female EM residents as observed by male and female faculty throughout residency and analyzed using multilevel mixed-effects linear regression modeling.ResultsA total of 33 456 direct-observation evaluations were collected from 359 EM residents (237 men [66.0%] and 122 women [34.0%]) by 285 faculty members (194 men [68.1%] and 91 women [31.9%]) during the study period. Female and male residents achieved similar milestone levels during the first year of residency. However, the rate of milestone attainment was 12.7% (0.07 levels per year) higher for male residents through all of residency (95% CI, 0.04-0.09). By graduation, men scored approximately 0.15 milestone levels higher than women, which is equivalent to 3 to 4 months of additional training, given that the average resident gains approximately 0.52 levels per year using our model (95% CI, 0.49-0.54). No statistically significant differences in scores were found based on faculty evaluator gender (effect size difference, 0.02 milestone levels; 95% CI for males, -0.09 to 0.11) or evaluator-evaluatee gender pairing (effect size difference, -0.02 milestone levels; 95% CI for interaction, -0.05 to 0.01).Conclusions and relevanceAlthough male and female residents receive similar evaluations at the beginning of residency, the rate of milestone attainment throughout training was higher for male than female residents across all EM subcompetencies, leading to a gender gap in evaluations that continues until graduation. Faculty should be cognizant of possible gender bias when evaluating medical trainees.

Dataset Information

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Unlabelled

Conclusions

Publications

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure