Dataset Information

Everyone Is Awesome: Analyzing Letters of Reference in a General Surgery Residency Selection Process.

ABSTRACT:

Background

The resident selection process involves the analysis of multiple data points, including letters of reference (LORs), which are inherently subjective in nature.

Objective

We assessed the frequency with which LORs use quantitative terms to describe applicants and to assess whether the use of these terms reflects the ranking of trainees in the final selection process.

Methods

A descriptive study analyzing LORs submitted by Canadian medical graduate applicants to the University of Ottawa General Surgery Program in 2019 was completed. We collected demographic information about applicants and referees and recorded the use of preidentified quantitative descriptors (eg, best, above average). A 10% audit of the data was performed. Descriptive statistics were used to analyze the demographics of our letters as well as the frequency of use of the quantitative descriptors.

Results

Three hundred forty-three LORs for 114 applicants were analyzed. Eighty-five percent (291 of 343) of LORs used quantitative descriptors. Eighty-four percent (95 of 113) of applicants were described as above average, and 45% (51 of 113) were described as the "best" by at least 1 letter. The candidates described as the "best" ranked anywhere from second to 108th in our ranking system.

Conclusions

Most LORs use quantitative descriptors. These terms are generally positive, and while the use does discriminate between different applicants, it was not helpful in the context of ranking applicants in our file review process.

SUBMITTER: Towaij C

PROVIDER: S-EPMC7594789 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Women are disproportionately underrecognized as award winners within medical societies. The presence of this disparity has not been investigated in training programs. To determine the presence of a gender disparity in award winners in general surgery residency programs. In this retrospective survey study, 32 geographically diverse academic and independent general surgery residency programs were solicited for participation. The 24 participating programs (75.0%) submitted deidentified data regarding the gender distribution of residents and trainee award recipients for the period from July 1, 1996, to June 30, 2017. Data were analyzed from September 11, 2017, to December 21, 2018. Time and the proportion of female trainees. The primary outcome was the percentage of female award winners. A multilevel logistic regression model accounting for the percentage of female residents in each program compared the odds of a female resident winning an award relative to a male resident. This analysis was repeated for the first and second decades of the study. Award winners were further analyzed by type of award (clinical excellence, nonclinical excellence, teaching, or research) and selection group (medical students, residents, or faculty members). A total of 5030 of 13 760 resident person-years (36.6%) and 455 of 1447 award winners (31.4%) were female. Overall, female residents were significantly less likely to receive an award compared with male residents (odds ratio [OR], 0.44; 95% CI, 0.37-0.54; P < .001). During the first decade of the study, female residents were 70.8% less likely to receive an award compared with male residents (OR, 0.29; 95% CI, 0.19-0.45; P < .001); this improved to 49.9% less likely in the second decade (OR, 0.50; 95% CI, 0.42-0.61; P < .001). Female residents were less likely to receive an award for teaching (OR, 0.33; 95% CI, 0.26-0.42; P < .001), clinical excellence (OR, 0.44; 95% CI, 0.31-0.61; P < .001), or nonclinical excellence (OR, 0.69; 95% CI, 0.48-0.98; P = .04). No statistical difference was observed for research award winners (OR, 0.76; 95% CI, 0.42-1.12; P = .17). The largest discrepancies were observed when award recipients were chosen by residents (OR, 0.23; 95% CI, 0.14-0.39; P < .001) and students (OR, 0.32; 95% CI, 0.25-0.42; P < .001) compared with faculty members (OR, 0.52; 95% CI, 0.42-0.66; P < .001). This study found that female residents were significantly underrepresented as award recipients. These findings suggest the presence of ongoing implicit bias in surgery departments and training programs.

Project description:BackgroundLetters of recommendation are considered one of the most important factors for whether an applicant is selected for an interview for orthopaedic surgery residency programs. Language differences in letters describing men versus women candidates may create differential perceptions by gender. Given the gender imbalance in orthopaedic surgery, we sought to determine whether there are differences in the language of letters of recommendation by applicant gender.Questions/purposes(1) Are there differences in word count and word categories in letters of recommendation describing women and men applicants, regardless of author gender? (2) Is author gender associated with word category differences in letters of recommendation? (3) Do authors of different academic rank use different words to describe women versus men applicants?MethodsUsing a linguistic analysis in a retrospective study, we analyzed all letters of recommendation (2834 letters) written for all 738 applicants with completed Electronic Residency Application Service applications submitted to the Johns Hopkins Orthopaedic Surgery Residency program during the 2018 to 2019 cycle to determine differences in word category use among applicants by gender, authors by gender, and authors by academic rank. Thirty nine validated word categories from the Linguistic Inquiry and Word Count dictionary along with seven additional word categories from previous publications were used in this analysis. The occurrence of words in each word category was divided by the number of words in the letter to obtain a word frequency for each letter. We calculated the mean word category frequency across all letters and analyzed means using non-parametric tests. For comparison of two groups, a p value threshold of 0.05 was used. For comparison of multiple groups, the Bonferroni correction was used to calculate an adjusted p value (p = 0.00058).ResultsLetters of recommendation for women applicants were slightly longer compared with those for men applicants (366 ± 188 versus 339 ± 199 words; p = 0.003). When comparing word category differences by applicant gender, letters for women applicants had slightly more "achieve" words (0.036 ± 0.015 versus 0.035 ± 0.018; p < 0.0001). Letters for men had more use of their first name (0.016 ± 0.013 versus 0.014 ± 0.009; p < 0.0001), and more "young" words (0.001 ± 0.003 versus 0.000 ± 0.001; p < 0.0001) than letters for women applicants. These differences were very small as each 0.001 difference in mean word frequency was equivalent to one more additional word from the word category appearing when comparing three letters for women to three letters for men. For differences in letters by author gender, there were no word category differences between men and women authors. Finally, when looking at author academic rank, letters for men applicants written by professors had slightly more "research" terms (0.011 ± 0.010) than letters written by associate professors (0.010 ± 0.010) or faculty of other rank (0.009 ± 0.011; p < 0.0001), a finding not observed in letters written for women.ConclusionsAlthough there were some minor differences favoring women, language in letters of recommendation to an academic orthopaedic surgery residency program were overall similar between men and women applicants.Clinical relevanceGiven the similarity in language between men and women applicants, increasing women applicants may be a more important factor in addressing the gender gap in orthopaedics.

Project description:Rationale and objectivesVirtual residency interviews have become crucial in maintaining CDC-recommended precautionary measures throughout the ongoing COVID-era. However, scant literature exists regarding the direct impact the pandemic has had on the radiology program selection process, including preferred modalities of residency interviews and social media utilization. This survey aimed to understand how radiology programs adapted to changes in the 2020-2021 virtual application cycle.Materials and methodsA 32-question survey was distributed to 194 residency programs between July and August of 2021. Data were analyzed primarily by using descriptive statistics and Paired Student's T-testing.ResultsA total of 31 programs completed the survey, yielding a response rate of 16%. When queried about the perceived success of virtual interviews during the 2020-2021 application cycle, 21 programs (68%) said the new interview format was very successful. Twenty-seven of the programs (87%) believed they were able to adequately gauge applicants through virtual interviews. However, when surveyed about personal preferences for interviews, the responses were more evenly distributed between virtual (11/31, 35%) and in-person (14/31, 45%). Pre-COVID, the top three criteria programs used to rank candidates were USMLE Step 1 score (20/31, 65%), letters of recommendation (17/31, 55%), and MSPE (12/31, 39%). Within the virtual, COVID-19 era, these criteria remained statistically unchanged (p = 0.22): USMLE Step 1 score (21/31, 68%), letters of recommendation (17/31, 55%), and MSPE (14/31, 45%). About half of programs who had not previously utilized social media (12 of 23, 52%) created accounts, mostly via Twitter or Instagram.ConclusionThe primary findings indicate that programs perceived success with virtually interviewing and ranking applicants, the criteria to rank applicants remain unchanged despite the virtual environment, and programs' social media utilization increased.

Project description:The COVID-19 pandemic has led to the cancellation of away rotations and other experiences fundamental to orthopaedic surgery residency application. Limited information is available on the experiences of applicants and program directors (PDs) during the COVID-19 pandemic residency application cycle. This study aimed to evaluate the current perspectives of applicants and PDs regarding the orthopaedic surgery residency application process. This was a cross-sectional study using 2 independent but similar surveys that addressed multiple aspects of the application process during the COVID-19 pandemic, including perceived effect of virtual experiences. Between February and March 2021, the surveys were distributed to orthopaedic surgery applicants and orthopaedic residency PDs. In total, 113 applicants (20.1%) and 29 PDs (19.6%) completed the survey. Applicants applied to 97.6 programs and received 13.3 interviews. They participated in 2.4 virtual away rotations. In total, 79.3% of programs reported offering some form of virtual opportunity, including virtual away rotations (24.0%), virtual happy hours (64.0%), and virtual conferences (64.0%). Programs offering virtual away rotations hosted 46.8 rotators and only invited back 54.5% for an interview. Applicants were most concerned about the lack of away rotations, the interview, and networking during this cycle, and 51% reported less confidence in matching. The most important factors for influencing applicant rank lists were perceived happiness of residents, resident camaraderie, and geographic location. However, residency program social events were not well replicated in a virtual setting. The COVID-19 pandemic presented new challenges for applicants and PDs. Applicants had less clinical exposure and received less interview invites after virtual away rotations. Despite applying to more programs, applicants received fewer interviews than in previous years. The virtual experiences adopted in this cycle did not adequately replicate the social factors that applicants found most important when ranking a program. Even during the COVID-19 pandemic, PDs most highly valued away rotation performance, clinical rotation performance, and board examination scores when offering interviews.