Dataset Information

Differences in Narrative Language in Evaluations of Medical Students by Gender and Under-represented Minority Status.

ABSTRACT:

Background

In varied educational settings, narrative evaluations have revealed systematic and deleterious differences in language describing women and those underrepresented in their fields. In medicine, limited qualitative studies show differences in narrative language by gender and under-represented minority (URM) status.

Objective

To identify and enumerate text descriptors in a database of medical student evaluations using natural language processing, and identify differences by gender and URM status in descriptions.

Design

An observational study of core clerkship evaluations of third-year medical students, including data on student gender, URM status, clerkship grade, and specialty.

Participants

A total of 87,922 clerkship evaluations from core clinical rotations at two medical schools in different geographic areas.

Main measures

We employed natural language processing to identify differences in the text of evaluations for women compared to men and for URM compared to non-URM students.

Key results

We found that of the ten most common words, such as "energetic" and "dependable," none differed by gender or URM status. Of the 37 words that differed by gender, 62% represented personal attributes, such as "lovely" appearing more frequently in evaluations of women (p < 0.001), while 19% represented competency-related behaviors, such as "scientific" appearing more frequently in evaluations of men (p < 0.001). Of the 53 words that differed by URM status, 30% represented personal attributes, such as "pleasant" appearing more frequently in evaluations of URM students (p < 0.001), and 28% represented competency-related behaviors, such as "knowledgeable" appearing more frequently in evaluations of non-URM students (p < 0.001).

Conclusions

Many words and phrases reflected students' personal attributes rather than competency-related behaviors, suggesting a gap in implementing competency-based evaluation of students. We observed a significant difference in narrative evaluations associated with gender and URM status, even among students receiving the same grade. This finding raises concern for implicit bias in narrative evaluation, consistent with prior studies, and suggests opportunities for improvement.

SUBMITTER: Rojek AE

PROVIDER: S-EPMC6502922 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:INTRODUCTION:Prior research demonstrates gender differences in language used in letters of recommendation. The emergency medicine (EM) Standardized Letter of Evaluation (SLOE) format limits word count and provides detailed instructions for writers. The objective of this study is to examine differences in language used to describe men and women applicants within the SLOE narrative. METHODS:All applicants to a four-year academic EM residency program within a single application year with a first rotation SLOE available were included in the sample. We used the Linguistic Inquiry and Word Count (LIWC) program to analyze word frequency within 16 categories. Descriptive statistics, chi-squared, and t-tests were used to describe the sample; gender differences in word frequency were tested for using Mann-Whitney U tests. RESULTS:Of 1117 applicants to the residency program, 822 (82%) first-rotation SLOEs were available; 64% were men, and 36% were women. We did not find a difference in baseline characteristics including age (mean 27 years), top 25 schools (22.5%), Alpha Omega Alpha Honor Medical Society rates (13%), and having earned advanced degrees (10%). The median word count per SLOE narrative for men was 171 and for women was 180 (p = 0.15). After adjusting for letter length, word frequency differences between genders were only present in two categories: social words (women: 23 words/letter; men: 21 words/letter, p = 0.02) and ability words (women: 2 words/letter; men: 1 word/letter, p = 0.04). We were unable to detect a statistical difference between men and women applicants in the remaining categories, including words representing communal traits, agentic traits, standout adjectives, grindstone traits, teaching words, and research words. CONCLUSION:The small wording differences between genders noted in two categories were statistically significant, but of unclear real-world significance. Future work is planned to evaluate how the SLOE format may contribute to this relative lack of bias compared to other fields and formats.

Project description:ImportanceWomen are underrepresented at higher ranks in academic medicine. However, the factors contributing to this disparity have not been fully elucidated. Implicit bias and unconscious mental attitudes toward a person or group may be factors. Although academic medical centers use physician trainee evaluations of faculty to inform promotion decisions, little is known about gender bias in these evaluations. To date, no studies have examined narrative evaluations of medical faculty by physician trainees for differences based on gender.ObjectiveTo characterize gender-associated linguistic differences in narrative evaluations of medical faculty written by physician trainees.Design, setting, and participantsThis retrospective cohort study included all faculty teaching evaluations completed for the department of medicine faculty by medical students, residents, and fellows at a large academic center in Pennsylvania from July 1, 2015, through June 30, 2016. Data analysis was performed from June 1, 2018, through July 31, 2018.Main outcomes and measuresWord use in faculty evaluations was quantified using automated text mining by converting free-text comments into unique 1- and 2-word phrases. Mixed-effects logistic regression analysis was performed to assess associations of faculty gender with frequencies of specific words and phrases present in a physician trainee evaluation.ResultsA total of 7326 unique evaluations were collected for 521 faculty (325 men [62.4%] and 196 women [37.6%]). The individual words art (odds ratio [OR], 7.78; 95% CI, 1.01-59.89), trials (OR, 4.43; 95% CI, 1.34-14.69), master (OR, 4.24; 95% CI, 1.69-10.63), and humor (OR, 2.32; 95% CI, 1.44-3.73) were significantly associated with evaluations of male faculty, whereas the words empathetic (OR, 4.34; 95% CI, 1.56-12.07), delight (OR, 4.26; 95% CI, 1.35-13.40), and warm (OR, 3.45; 95% CI, 1.83-6.49) were significantly associated with evaluations of female faculty. Two-word phrases associated with male faculty evaluations included run rounds (OR, 7.78; 95% CI, 1.01-59.84), big picture (OR, 7.15; 95% CI, 1.68-30.42), and master clinician (OR, 4.02; 95% CI, 1.21-13.36), whereas evaluations of female faculty were more likely to be associated with model physician (OR, 7.75; 95% CI, 1.70-35.39), just right (OR, 6.97; 95% CI, 1.51-32.30), and attention (to) detail (OR, 4.26; 95% CI, 1.36-13.40).Conclusions and relevanceThe data showed quantifiable linguistic differences between free-text comments about male and female faculty in physician trainee evaluations. Further evaluation of these differences, particularly in association with ongoing gender disparities in faculty promotion and retention, may be warranted.

Project description:BackgroundThe shortage of doctors and maldistribution among specialties are of great concern in the Japanese health care system. This study investigated specialty preference in medical students of one university, and examined gender differences and compared their preference with real needs.MethodsWe conducted a self-administered questionnaire including specialty preference in all students of one medical university. Preference was assessed by the five-level probability of their future choice: 1 = very low, 2 = low, 3 = moderate, 4 = high, and 5 = very high. The proportion of 4 or 5 was calculated as the preference rate. The real needs (magnitude of doctor shortage) in the prefecture were drawn from two different surveys. The relationship between the sex-specific preference rate by specialty and real needs was assessed by Spearman's correlation coefficient.ResultsInternal medicine showed the highest preference rate, followed by general surgery, pediatrics, and emergency medicine. There was no significant correlation between the preference rates of men and women (r = 0.27, p = 0.34). The preference rates for general surgery, orthopedics, neurosurgery, and emergency medicine were significantly higher in men than in women, while those of obstetrics & gynecology, pediatrics, and dermatology were significantly higher in women. The magnitude of doctor shortage by specialty from two surveys were significantly correlated with the total preference rate and men's preference rate (r = 0.54 to 0.74), but not with women's preference rate (r = 0.06 and 0.32).ConclusionsThis study elucidated not only gender differences in specialty preference but also the relationship to real needs. Critical gender differences and mismatch with real needs were found in women. In addition to traditional gender roles and insufficient support for women's participation in Japan, gender differences and mismatch influence the current and future maldistribution of specialties. Systematic changes in the working environment in medical society are required to solve these problems.

Project description:Mistreatment of medical students during medical education is a widespread concern. Studies have shown that medical students report the most mistreatment compared to students of other study programs and that the prevalence of mistreatment peaks during clinical training. For this reason, a study was conducted to assess prevalence of mistreatment among medical students committed by various groups of people. The focus was to identify whether gender was associated with the experience of mistreatment. Additionally, students' perception of university climate for reporting sexual harassment was assessed.In the study 88 medical students (45 women, 43 men) participated. A modified version of the Questionnaire on Student Abuse was used to assess students' experience of various types of mistreatment and associated distress during medical education. To explore factors that could be associated with this experience the organizational climate for reporting sexual harassment was assessed with the Psychological Climate for Sexual Harassment.The most often cited perpetrators of mistreatment were strangers (79.5%), friends (75.0%) and university staff (68.2%). Strangers mostly committed psychological mistreatment and sexual harassment, whereas friends additionally engaged in physical mistreatment of medical students. The most common form of mistreatment conducted by university staff was humiliation of students. These kinds of psychological mistreatment were reported to be distressing (43%). Gender differences were found in the prevalence of mistreatment. Women experienced more sexual harassment and humiliation than did men. On the other hand, men experienced more physical mistreatment than did women. Women reported experiencing more distress from mistreatment experiences than did men and also more often reported being mistreated by university staff than did men. Women perceived a greater risk in reporting sexual harassment to the organization than did men.Mistreatment of female and male students should be focused on using a gender perspective because types of mistreatment can differ by gender. Additionally, interventions should include the societal level as there was a high prevalence of mistreatment perpetrated by strangers. Also the issue of trust in the university needs to be addressed and the organization is called on to visibly demonstrate that it represents and protects its students as well as its staff.