Dataset Information

Dear Program Director: Deciphering Letters of Recommendation.

ABSTRACT:

Background?

Letters of recommendation (LORs) are an important part of applications for residency and fellowship programs. Despite anecdotal use of a "code" in LORs, research on program director (PD) perceptions of the value of these documents is sparse.

Objective?

We analyzed PD interpretations of LOR components and discriminated between perceived levels of applicant recommendations.

Methods?

We conducted a cross-sectional, descriptive study of pediatrics residency and fellowship PDs. We developed a survey asking PDs to rate 3 aspects of LORs: 13 letter features, 10 applicant abilities, and 11 commonly used phrases, using a 5-point Likert scale. The 11 phrases were grouped using principal component analysis. Mean scores of components were analyzed with repeated-measures analysis of variance. Median Likert score differences between groups were analyzed with Mann-Whitney U tests.

Results?

Our survey had a 43% response rate (468 of 1079). "I give my highest recommendation" was rated the most positive phrase, while "showed improvement" was rated the most negative. Principal component analysis generated 3 groups of phrases with moderate to strong correlation with each other. The mean Likert score for each group from the PD rating was calculated. Positive phrases had a mean (SD) of 4.4 (0.4), neutral phrases 3.4 (0.5), and negative phrases 2.6 (0.6). There was a significant difference among all 3 pairs of mean scores (all P < .001).

Conclusions?

Commonly used phrases in LORs were interpreted consistently by PDs and influenced their impressions of candidates. Key elements of LORs include distinct phrases depicting different degrees of endorsement.

SUBMITTER: Saudek K

PROVIDER: S-EPMC6008019 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Dear Program Director: Deciphering Letters of Recommendation.

Saudek Kris K Saudek David D Treat Robert R Bartz Peter P Weigert Rachel R Weisgerber Michael M

Journal of graduate medical education 20180601 3

<h4>Background</h4>Letters of recommendation (LORs) are an important part of applications for residency and fellowship programs. Despite anecdotal use of a "code" in LORs, research on program director (PD) perceptions of the value of these documents is sparse.<h4>Objective</h4>We analyzed PD interpretations of LOR components and discriminated between perceived levels of applicant recommendations.<h4>Methods</h4>We conducted a cross-sectional, descriptive study of pediatrics residency and fellows ...[more]

PMID: 29946380

Similar Datasets

Project description:BackgroundLetters of recommendation are considered one of the most important factors for whether an applicant is selected for an interview for orthopaedic surgery residency programs. Language differences in letters describing men versus women candidates may create differential perceptions by gender. Given the gender imbalance in orthopaedic surgery, we sought to determine whether there are differences in the language of letters of recommendation by applicant gender.Questions/purposes(1) Are there differences in word count and word categories in letters of recommendation describing women and men applicants, regardless of author gender? (2) Is author gender associated with word category differences in letters of recommendation? (3) Do authors of different academic rank use different words to describe women versus men applicants?MethodsUsing a linguistic analysis in a retrospective study, we analyzed all letters of recommendation (2834 letters) written for all 738 applicants with completed Electronic Residency Application Service applications submitted to the Johns Hopkins Orthopaedic Surgery Residency program during the 2018 to 2019 cycle to determine differences in word category use among applicants by gender, authors by gender, and authors by academic rank. Thirty nine validated word categories from the Linguistic Inquiry and Word Count dictionary along with seven additional word categories from previous publications were used in this analysis. The occurrence of words in each word category was divided by the number of words in the letter to obtain a word frequency for each letter. We calculated the mean word category frequency across all letters and analyzed means using non-parametric tests. For comparison of two groups, a p value threshold of 0.05 was used. For comparison of multiple groups, the Bonferroni correction was used to calculate an adjusted p value (p = 0.00058).ResultsLetters of recommendation for women applicants were slightly longer compared with those for men applicants (366 ± 188 versus 339 ± 199 words; p = 0.003). When comparing word category differences by applicant gender, letters for women applicants had slightly more "achieve" words (0.036 ± 0.015 versus 0.035 ± 0.018; p < 0.0001). Letters for men had more use of their first name (0.016 ± 0.013 versus 0.014 ± 0.009; p < 0.0001), and more "young" words (0.001 ± 0.003 versus 0.000 ± 0.001; p < 0.0001) than letters for women applicants. These differences were very small as each 0.001 difference in mean word frequency was equivalent to one more additional word from the word category appearing when comparing three letters for women to three letters for men. For differences in letters by author gender, there were no word category differences between men and women authors. Finally, when looking at author academic rank, letters for men applicants written by professors had slightly more "research" terms (0.011 ± 0.010) than letters written by associate professors (0.010 ± 0.010) or faculty of other rank (0.009 ± 0.011; p < 0.0001), a finding not observed in letters written for women.ConclusionsAlthough there were some minor differences favoring women, language in letters of recommendation to an academic orthopaedic surgery residency program were overall similar between men and women applicants.Clinical relevanceGiven the similarity in language between men and women applicants, increasing women applicants may be a more important factor in addressing the gender gap in orthopaedics.

Project description:Abstract Objective Application for otolaryngology residency is highly competitive, with letters of recommendation (LORs) and applicant personal statements (PSs) representing important components of the application process. However, their inherently subjective nature predisposes them to potential implicit bias. Otolaryngology has historically been predominated by male physicians and while implicit sex bias has been demonstrated in LORs for application to residency of multiple specialties, data is limited for otolaryngology. Methods LORs and PSs for all otolaryngology applicants to an academic medical center during the 2019–20 and 2020–21 cycles were abstracted. Quantitative analysis was performed using Linguistic Inquiry and Word Count 2015 (LIWC2015), a validated software application designed to analyze various emotional, cognitive, and structural components of written text. Results LORs written for females were found to be written from a perspective of higher expertise and confidence while LORs written for males were associated with a more honest, personal, and disclosing tone. Moreover, LORs written for female applicants were found to reference achievement and “grindstone” terminology more than those written for men. No differences were observed in any word category between PSs written by male and female applicants. Conclusion Minor linguistic differences exist in multiple domains between LORs written for male and female applicants for otolaryngology residency. These tended to favor female applicants, with their letters demonstrating higher clout, achievement, and grindstone scores. This trend was unexpected in this historically predominantly male specialty. While differences were statistically significant, the overall difference in an entire letter of recommendation is likely subtle. Level of Evidence 3. Implicit sex bias has been demonstrated in letters of recommendation for application to residency of multiple specialties; however, data is limited for otolaryngology. Letters of recommendation and personal statements for all otolaryngology applicants to an academic medical center during the 2019–20 and 2020–21 cycles were analyzed using Linguistic Inquiry and Word Count 2015, a validated software application designed to analyze various emotional, cognitive, and structural components of written text. Minor linguistic differences exist in multiple domains between LORs written for male and female applicants for otolaryngology residency.

Project description:Background: This survey aims to identify the relative value and the critical components of anesthesiology letters of recommendation(LORs) from the perspective of Program Directors (PDs) and Associate/Assistant Program Directors (APDs). Knowledge and insights originating from this survey might add to the understanding of the anesthesiology residency selection process and mitigate unintended linguistic biases.Methodology: Anonymous online surveys were sent to anesthesiology PDs/APDs from the Accreditation Council for Graduate Medical Education (ACGME) accredited anesthesiology residency Programs in the USA (US), as listed on the ACGME website and the American Medical Association Fellowship and Residency Electronic Interactive Database (AMA FREIDA) Residency Program Database. The survey authors were blinded to the identity of the respondents.Results: 62 out of 183 (33.8%) invited anesthesiology PDs/APDs completed the survey anonymously. In our survey, LORs are reported as more important in granting an interview than in making the rank list. 64% of respondents prefer narrative LORs. 77.4% of respondents look for specific keywords in LORs. Keywords such as 'top % of students' and 'we are recruiting this candidate' indicate a strong letter of recommendation while keywords such as 'I recommend to your program' or non-superlative descriptions indicate a weak letter of recommendation. Other key components of LORs include the specialty of the letter-writer, according to 84% of respondents, with anesthesiology as the most valuable specialty. Although narrative LORs are preferred, 55.1% of respondents are not satisfied with the content of narrative LORs.Conclusion: LORs containing specific keywords play an important role in the application to anesthesiology residency, particularly when submitted by an anesthesiologist. While narrative LORs are still the preferred format, most of our respondents feel they need improvements. The authors suggest specific LOR improvements including creating formalized LOR training, adding a style guide, and applying comparative scales, with standardized vocabulary in the narrative LOR.

Project description:BackgroundThe last formal review of academic anesthesiology department Web sites (ADWs) for content was conducted in 2009. ADWs have been rated as very important by medical students in researching residency training programs; however, the rapid evolution of sites require that descriptive statistics must be more current to be considered reliable. We set out to provide an updated overview of ADW content and to better understand residency program directors' (PD) role and comfort with ADWs.MethodsTwo independent reviewers (ND and HL) analyzed all 131 Accreditation Council for Graduate Medical Education (ACGME) accredited ADWs. A binary system (Yes/No) was used to determine which features were present. Reviewer reliability was confirmed with inter-rater reliability and percentage agreement calculation. Additionally, a blinded electronic survey (Survey Monkey, Portland, OR) was sent to anesthesiology residency PDs via electronic mail investigating the audiences for ADWs, the frequency of updates and the degree of PD involvement.Results13% of anesthesiology departments still lack a Web site with a homepage with links to the residency program and educational offerings (18% in 2009). Only half (55%) of Web sites contain information for medical students, including clerkship information. Furthermore, programs rarely contain up-to-date calendars (13%), accreditation cycle lengths (11%), accreditation dates (7%) or board pass rates (6%). The PD survey, completed by 42 of 131 PDs, noted a correlation (r = 0.36) between the number of years as PD and the frequency of Web site updates - less experienced PDs appear to update their sites more frequently (p = 0.03). Although 86% of PDs regarded a Web site as "very" important in recruitment, only 9% felt "very" comfortable with the skills required to advertise and market a Web site.ConclusionsDespite the overall increase in ADW content since 2009, privacy concerns, limited resources and time constraints may prevent PDs from providing the most up-to-date Web sites for applicants and other interested audiences. PDs are aware of value of Web sites for recruitment, are typically involved in determining ADW content, but few feel very comfortable marketing a training program on the Web.

Project description:IntroductionThe persistent lack of racial and ethnic diversity within the field of otolaryngology calls for an analysis of potential bias within the residency application system. Letters of recommendation (LORs) and personal statements (PSs) are the most important subjective application constituents. This subjectivity predisposes these components to implicit bias. In applications to various surgical subspecialties, prior linguistic studies assessing bias in reviews of LOR show race-based differences. Thus far, racial and ethnic linguistic differences in LORs for otolaryngology applicants have not been analyzed in the literature.MethodsLORs and PSs were abstracted from otolaryngology - head and neck surgery applications in the Electronic Residency Application Service for the 2019-20 and 2020-21 application cycles. Linguistic Inquiry and Word Count 2015 was used for quantitative analysis of emotional, cognitive, and structural components of written text.ResultsRace-pair analysis of the 2019-2021 application cycles revealed higher mean "teaching" scores for LORs for Asian, black, Hispanic, and white applicants when compared to applicants who self-identified as Other. White applicants had lower scores for the terms "research" and "analytic" when compared to Asian and black applicants, respectively. Analysis of PSs revealed greater scores for an "authentic" writing style for white versus Asian applicants. White applicants were found to have higher scores for "tone" compared to black applicants.ConclusionMinor racial and ethnic language differences exist in both LORs and PSs. A statistically significant difference was observed among LORs, with the "teaching" term used more frequently for Asian, black, Hispanic, and white applicants compared to self-identified Other individuals. For PSs, statistically significant differences were observed among white applicants, who wrote about themselves using more "authentic" language when compared with Asian applicants and who also had higher scores for "tone" compared to black applicants. Although these differences were statistically significant, the practical impact of the variances is likely small.

Dataset Information

Dear Program Director: Deciphering Letters of Recommendation.

Background?

Objective?

Methods?

Results?

Conclusions?

Publications

Dear Program Director: Deciphering Letters of Recommendation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets