Dataset Information

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director.

ABSTRACT:

Objective

This study sought to evaluate the perspectives of surgical program directors regarding the change of USMLE Step 1 to pass/fail grading.

Design

Validated electronic survey.

Setting

Vanderbilt University Medical Center Department of Plastic Surgery.

Participants

Program directors of all ACMGE-accredited General Surgery, Integrated Vascular Surgery, Integrated Thoracic Surgery, and Integrated Plastic Surgery residency programs.

Results

The overall response rate was 55.5%. Most PDs (78.1%) disagreed with the scoring change. Only 19.6% believe this change will improve medical student well-being. For 63.5% of PDs, medical school pedigree will become more important, and 52.7% believe it will place international medical graduates at a disadvantage. Only 6.2% believe Step 2 CK should also be pass/fail, while 88.7% will increase the weight of Step 2 CK and 88.4% will now require Step 2 CK score submission with the electronic residency application service.

Conclusions

While well-intentioned, changing USMLE Step 1 to pass/fail may have unintended consequences and may disadvantage certain groups of applicants. The emphasis on Step 1, and resulting test-taking apprehension, will likely shift to Step 2 CK. Proponents of equitable evaluation should direct their efforts toward increasing, not decreasing, the number of objective measures available for student assessment.

SUBMITTER: Pontell ME

PROVIDER: S-EPMC7347473 | biostudies-literature | 2021 Jan-Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director.

Pontell Matthew E ME Makhoul Alan T AT Ganesh Kumar Nishant N Drolet Brian C BC

Journal of surgical education 20200710 1

<h4>Objective</h4>This study sought to evaluate the perspectives of surgical program directors regarding the change of USMLE Step 1 to pass/fail grading.<h4>Design</h4>Validated electronic survey.<h4>Setting</h4>Vanderbilt University Medical Center Department of Plastic Surgery.<h4>Participants</h4>Program directors of all ACMGE-accredited General Surgery, Integrated Vascular Surgery, Integrated Thoracic Surgery, and Integrated Plastic Surgery residency programs.<h4>Results</h4>The overall respo ...[more]

PMID: 32654997

Similar Datasets

Project description:BackgroundUnited States Medical Licensing Examination Step 1 will transition from numeric grading to pass/fail, sometime after January 2022. The aim of this study was to compare how program directors in orthopaedics and internal medicine perceive a pass/fail Step 1 will impact the residency application process.MethodsA 27-item survey was distributed through REDCap to 161 U.S. orthopaedic residency program directors and 548 U.S. internal medicine residency program directors. Program director emails were obtained from the American Medical Association's Fellowship and Residency Electronic Interactive Database.ResultsWe received 58 (36.0%) orthopaedic and 125 (22.8%) internal medicine program director responses. The majority of both groups disagree with the change to pass/fail, and felt that the decision was not transparent. Both groups believe that the Step 2 Clinical Knowledge exam and clerkship grades will take on more importance. Compared to internal medicine PDs, orthopaedic PDs were significantly more likely to emphasize research, letters of recommendation from known faculty, Alpha Omega Alpha membership, leadership/extracurricular activities, audition elective rotations, and personal knowledge of the applicant. Both groups believe that allopathic students from less prestigious medical schools, osteopathic students, and international medical graduates will be disadvantaged. Orthopaedic and internal medicine program directors agree that medical schools should adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residency applications a student can submit.ConclusionOrthopaedic and internal medicine program directors disagree with the change of Step 1 to pass/fail. They also believe that this transition will make the match process more difficult, and disadvantage students from less highly-regarded medical schools. Both groups will rely more heavily on the Step 2 clinical knowledge exam score, but orthopaedics will place more importance on research, letters of recommendation, Alpha Omega Alpha membership, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives.

Project description:BACKGROUND: To assess the impact of a change in preclerkship grading system from Honors/Pass/Fail (H/P/F) to Pass/Fail (P/F) on University of California, San Diego (UCSD) medical students' academic performance. METHODS: Academic performance of students in the classes of 2011 and 2012 (constant-grading classes) were collected and compared with performance of students in the class of 2013 (grading-change class) because the grading policy at UCSD SOM was changed for the class of 2013, from H/P/F during the first year (MS1) to P/F during the second year (MS2). For all students, data consisted of test scores from required preclinical courses from MS1 and MS2 years, and USMLE Step 1 scores. Linear regression analysis controlled for other factors that could be predictive of student performance (i.e., MCAT scores, undergraduate GPA, age, gender, etc.) in order to isolate the effect of the changed grading policy on academic performance. The change in grading policy in the MS2 year only, without any corresponding changes to the medical curriculum, presents a unique natural experiment with which to cleanly evaluate the effect of P/F grading on performance outcomes. RESULTS: After controlling for other factors, the grading policy change to P/F grading in the MS2 year had a negative impact on second-year grades relative to first-year grades (the constant-grading classes performed 1.65% points lower during their MS2 year compared to the MS1 year versus 3.25% points lower for the grading-change class, p?<?0.0001), but had no observable impact on USMLE Step 1 scores. CONCLUSIONS: A change in grading from H/P/F grading to P/F grading was associated with decreased performance on preclinical examinations but no decrease in performance on the USMLE Step 1 examination. These results are discussed in the broader context of the multitude of factors that should be considered in assessing the merits of various grading systems, and ultimately the authors recommend the continuation of pass-fail grading at UCSD School of Medicine.

Project description:BackgroundExamination of arthroscopic skill requires evaluation tools that are valid and reliable with clear criteria for passing. The Arthroscopic Surgery Skill Evaluation Tool was developed as a video-based assessment of technical skill with criteria for passing established by a panel of experts. The purpose of this study was to test the validity and reliability of the Arthroscopic Surgery Skill Evaluation Tool as a pass-fail examination of arthroscopic skill.MethodsTwenty-eight residents and two sports medicine faculty members were recorded performing diagnostic knee arthroscopy on a left and right cadaveric specimen in our arthroscopic skills laboratory. Procedure videos were evaluated with use of the Arthroscopic Surgery Skill Evaluation Tool by two raters blind to subject identity. Subjects were considered to pass the Arthroscopic Surgery Skill Evaluation Tool when they attained scores of ≥ 3 on all eight assessment domains.ResultsThe raters agreed on a pass-fail rating for fifty-five of sixty videos rated with an interclass correlation coefficient value of 0.83. Ten of thirty participants were assigned passing scores by both raters for both diagnostic arthroscopies performed in the laboratory. Receiver operating characteristic analysis demonstrated that logging more than eighty arthroscopic cases or performing more than thirty-five arthroscopic knee cases was predictive of attaining a passing Arthroscopic Surgery Skill Evaluation Tool score on both procedures performed in the laboratory.ConclusionsThe Arthroscopic Surgery Skill Evaluation Tool is valid and reliable as a pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory.Clinical relevanceThis study demonstrates that the Arthroscopic Surgery Skill Evaluation Tool may be a useful tool for pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory. Further study is necessary to determine whether the Arthroscopic Surgery Skill Evaluation Tool can be used for the assessment of multiple arthroscopic procedures and whether it can be used to evaluate arthroscopic procedures performed in the operating room.

Project description:ImportancePrevious studies of resident attrition have variably included preliminary residents and likely overestimated categorical resident attrition. Whether program director attitudes affect attrition has been unclear.ObjectivesTo determine whether program director attitudes are associated with resident attrition and to measure the categorical resident attrition rate.Design, setting, and participantsThis multicenter study surveyed 21 US program directors in general surgery about their opinions regarding resident education and attrition. Data on total resident complement, demographic information, and annual attrition were collected from the program directors for the study period of July 1, 2010, to June 30, 2015. The general surgery programs were chosen on the basis of their geographic location, previous collaboration with some coauthors, prior work in surgical education and research, or a program director willing to participate. Only categorical surgical residents were included in the study; thus, program directors were specifically instructed to exclude any preliminary residents in their responses.Main outcomes and measuresFive-year attrition rates (2010-2011 to 2014-2015 academic years) as well as first-time pass rates on the General Surgery Qualifying Examination and General Surgery Certifying Examination of the American Board of Surgery (ABS) were collected. High- and low-attrition programs were compared.ResultsThe 21 programs represented different geographic locations and 12 university-based, 3 university-affiliated, and 6 independent program types. Programs had a median (interquartile range [IQR]) number of 30 (20-48) categorical residents, and few of those residents were women (median [IQR], 12 [5-17]). Overall, 85 of 966 residents (8.8%) left training during the study period: 15 (17.6%) left after postgraduate year 1, 34 (40.0%) after postgraduate year 2, and 36 (42.4%) after postgraduate year 3 or later. Forty-four residents (51.8%) left general surgery for another surgical discipline, 21 (24.7%) transferred to a different surgery program, and 18 (21.2%) exited graduate medical education altogether. Each program had an annual attrition rate ranging from 0.73% to 6.0% (median [IQR], 2.5% [1.5%-3.4%]). Low-attrition programs were more likely than high-attrition programs to use resident remediation (21.0% vs 6.8%; P?<?.001). Median (IQR) Qualifying Examination pass rates (93% [90%-98%] vs 92% [86%-100%]; P?=?.92) and Certifying Examination pass rates (83% [68%-84%] vs 81% [71%-86%]; P?=?.47) were similar. Program directors at high-attrition programs were more likely than their counterparts at low-attrition programs to agree with this statement: "I feel that it is my responsibility as a program director to redirect residents who should not be surgeons."Conclusions and relevanceThe overall 5-year attrition rate of 8.8% was significantly lower than previously reported. Program directors at low-attrition programs were more likely to use resident remediation. Variations in attrition may be explained by program director attitudes, although larger studies are needed to further define program factors affecting attrition.

Dataset Information

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director.

Objective

Design

Setting

Participants

Results

Conclusions

Publications

The Change of USMLE Step 1 to Pass/Fail: Perspectives of the Surgery Program Director.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets