Dataset Information

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

ABSTRACT:

Unlabelled

Background?From 2010 to 2011, more than 70% of the clinical rotation competency evaluations for nephrology fellows in our program were rated "superior" using a 9-point Likert scale, suggesting some degree of "grade inflation." Objective?We sought to assess the efficacy of a 5-point centered rotation evaluation in reducing grade inflation. Methods?This retrospective cohort study of the impact of faculty education and a 5-point rotation evaluation on grade inflation was measured by superior item rating frequency and proportion of evaluations without superior ratings. The 5-point evaluation centered performance at the level expected for stage of training. Faculty education began in 2011-2012. The 5-point centered evaluation was introduced in 2012-2013 and used exclusively thereafter. A total of 68 evaluations, using the 9-point Likert scale, and 63 evaluations, using the 5-point centered scale, were performed after first-year fellow clinical rotations. Nine to 12 faculty members participated yearly. Results?Faculty education alone was associated with fewer superior ratings from 2010-2011 to 2011-2012 (70.5% versus 48.3%, P?=?.001), declining further with 5-point centered scale introduction (2012-2013; 48.3% versus 35.6%; P?=?.012). Superior ratings declined with 5-point centered versus 9-point Likert scales (37.3% versus 59.3%, P?=?.001), specifically for medical knowledge, patient care, practice-based learning and improvement, and professionalism. On logistic regression, evaluations without superior scores were more likely for 5-point centered versus 9-point Likert scales (adjusted odds ratio [aOR]?=?8.26; 95% CI 1.53-44.64; P?=?.014) and associated with faculty identifier (aOR= 1.18; 95% CI 1.03-1.35; P?=?.013), but not fellow identifier or training year quarter.

Conclusions

Grade inflation was reduced with faculty education and the 5-point centered evaluation scale.

SUBMITTER: Yuan CM

PROVIDER: S-EPMC4857519 | biostudies-literature | 2016 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Yuan Christina M CM Nee Robert R Abbott Kevin C KC Oliver James D JD

Journal of graduate medical education 20160501 2

<h4>Unlabelled</h4>Background From 2010 to 2011, more than 70% of the clinical rotation competency evaluations for nephrology fellows in our program were rated "superior" using a 9-point Likert scale, suggesting some degree of "grade inflation." Objective We sought to assess the efficacy of a 5-point centered rotation evaluation in reducing grade inflation. Methods This retrospective cohort study of the impact of faculty education and a 5-point rotation evaluation on grade inflation was measured ...[more]

PMID: 27168886

Similar Datasets

Project description:Rationale & objectiveAdoption of point-of-care ultrasound (POCUS) into nephrology practice has been relatively slow. We surveyed US nephrology program directors, their fellows, and graduates from a single training program regarding current/planned POCUS training, clinical use, and barriers to training and use.Study designAnonymous, online survey.Setting & participantsAll US nephrology program directors (n=151), their fellows (academic year 2021-2022), and 89/90 graduates (1980-2021) of the Walter Reed Nephrology Program.Analytical approachDescriptive.Results46% (69/151) of program directors and 33% (118/361) of their fellows responded. Response rate was 62% (55/89) for Walter Reed graduates. 51% of program directors offered POCUS training, most commonly bedside training in non-POCUS oriented rotations (71%), didactic lectures (68%), and simulation (43%). 46% of fellows reported receiving POCUS training, but of these, many reported not being sufficiently trained/not confident in kidney (56%), bladder (50%), and inferior vena cava assessment (46%). Common barriers to training reported by program directors were not enough trained faculty (78%), themselves not being sufficiently trained (55%), and equipment expense (51%). 64% of program directors and 55% of fellows reported <10% of faculty were able to perform POCUS. 64% of fellows reported having too little POCUS training. 72% of program directors and 77% of graduates felt POCUS should be incorporated into the fellowship curriculum. 59% of fellows and 61% of graduates desired hands-on POCUS training rather than didactic lectures or simulation.LimitationsLoss of respondents as program directors and fellows progressed through the survey.ConclusionsNephrology program directors, fellows, and graduates surveyed want POCUS training incorporated into the fellowship curriculum. No group felt sufficiently trained to confidently perform POCUS, and the major barrier to training was lack of sufficiently trained faculty. This highlights the need to "train the trainers" before POCUS can be fully integrated into fellowship training and regularly used in nephrology practice.

Project description:ImportanceAlthough implicit bias in medical training has long been suspected, it has been difficult to study using objective measures, and the influence of sex and gender in the evaluation of medical trainees is unknown. The emergency medicine (EM) milestones provide a standardized framework for longitudinal resident assessment, allowing for analysis of resident performance across all years and programs at a scope and level of detail never previously possible.ObjectiveTo compare faculty-observed training milestone attainment of male vs female residency training.Design, setting, and participantsThis multicenter, longitudinal, retrospective cohort study took place at 8 community and academic EM training programs across the United States from July 1, 2013, to July 1, 2015, using a real-time, mobile-based, direct-observation evaluation tool. The study examined 33 456 direct-observation subcompetency evaluations of 359 EM residents by 285 faculty members.Main outcomes and measuresMilestone attainment for male and female EM residents as observed by male and female faculty throughout residency and analyzed using multilevel mixed-effects linear regression modeling.ResultsA total of 33 456 direct-observation evaluations were collected from 359 EM residents (237 men [66.0%] and 122 women [34.0%]) by 285 faculty members (194 men [68.1%] and 91 women [31.9%]) during the study period. Female and male residents achieved similar milestone levels during the first year of residency. However, the rate of milestone attainment was 12.7% (0.07 levels per year) higher for male residents through all of residency (95% CI, 0.04-0.09). By graduation, men scored approximately 0.15 milestone levels higher than women, which is equivalent to 3 to 4 months of additional training, given that the average resident gains approximately 0.52 levels per year using our model (95% CI, 0.49-0.54). No statistically significant differences in scores were found based on faculty evaluator gender (effect size difference, 0.02 milestone levels; 95% CI for males, -0.09 to 0.11) or evaluator-evaluatee gender pairing (effect size difference, -0.02 milestone levels; 95% CI for interaction, -0.05 to 0.01).Conclusions and relevanceAlthough male and female residents receive similar evaluations at the beginning of residency, the rate of milestone attainment throughout training was higher for male than female residents across all EM subcompetencies, leading to a gender gap in evaluations that continues until graduation. Faculty should be cognizant of possible gender bias when evaluating medical trainees.

Dataset Information

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Unlabelled

Conclusions

Publications

Combating Grade Inflation in Nephrology Clinical Rotation Evaluations Using Faculty Education and a 5-Point Centered Rating Scale.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets