Dataset Information

Effects of gender in resident evaluations and certifying examination pass rates.

ABSTRACT:

Background

Though the proportion of female Internal Medicine (IM) residents and faculty has increased, there is minimal large scale modern data comparing resident performance by gender. This study sought to examine the effects of resident and faculty gender on resident evaluations.

Methods

Retrospective observational study over 5?years in a single IM program. IM certifying examination pass rates were obtained from the American Board of IM.

Results

Four hundred eighty-eight residents (195 women, 293 men), evaluated by 430 attending physicians (163 women, 270 men) were included. Twelve thousand six hundred eighty-one evaluations between 2007 and 2012 were analyzed. Female residents scored higher in two domains (Medical Interviewing, and Interpersonal and Communication Skills) (p?ConclusionsData from one large academic medical center demonstrate higher ratings for female residents on performance domains reflecting bedside care and interpersonal skills, with similar scores for medical knowledge and remaining domains. No significant difference was seen locally in certifying examination scores, nor in recent national pass rates, an objective measure of medical knowledge. Despite imbalanced female representation in areas of medicine, our data suggest that gender-based disparities in Internal Medicine resident medical knowledge and physician competency are no longer present.

SUBMITTER: Sulistio MS

PROVIDER: S-EPMC6322320 | biostudies-literature | 2019 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effects of gender in resident evaluations and certifying examination pass rates.

Sulistio Melanie S MS Khera Amit A Squiers Kathryn K Sanghavi Monika M Ayers Colby R CR Weng Weifeng W Kazi Salahuddin S de Lemos James J Johnson David H DH Kirk Lynne L

BMC medical education 20190107 1

<h4>Background</h4>Though the proportion of female Internal Medicine (IM) residents and faculty has increased, there is minimal large scale modern data comparing resident performance by gender. This study sought to examine the effects of resident and faculty gender on resident evaluations.<h4>Methods</h4>Retrospective observational study over 5 years in a single IM program. IM certifying examination pass rates were obtained from the American Board of IM.<h4>Results</h4>Four hundred eighty-eight ...[more]

PMID: 30616651

Similar Datasets

Project description:Importance:American Board of Surgery board certification requires passing both a written qualifying examination and an oral certifying examination. No studies have been conducted assessing the effect of sociodemographic variables on board passage rates. Objective:To evaluate if trainee sociodemographic factors are associated with board passage rates. Design, Setting, and Participants:This national and multi-institutional prospective observational cohort study of 1048 categorical general surgery trainees starting in 2007-2008 were surveyed. Data collection began in June 2007, follow-up was completed on December 31, 2016, and analysis began September 2018. Main Outcomes and Measures:Survey responses were linked to American Board of Surgery board passage data. Results:Of 662 examinees who had complete survey and follow-up data, 443 (65%) were men and 459 (69%) were white, with an overall board passage rate of 87% (n?=?578). In a multinomial regression model, trainees of Hispanic ethnicity were more likely to not attempt the examinations (vs passed both) than non-Hispanic trainees (odds ratio [OR], 4.7; 95% CI, 1.5-14). Compared with examinees who were married with children during internship, examinees who were married without children (OR, 0.3; 95% CI, 0.1-0.8) or were single (OR, 0.4; 95% CI, 0.2-0.9) were less likely to fail the examinations. Logistic regression showed white examinees compared with nonwhite examinees (black individuals, Asian individuals, and individuals of other races) (OR, 1.8; 95% CI, 1.03-3.0) and examinees who performed better on their first American Board of Surgery In-Training Examination (OR, 1.03; 95% CI, 1.02-1.05) were more likely to pass the qualifying examination on the first try. White examinees compared with nonwhite examinees (OR, 1.8; 95% CI, 1.1-2.8), non-Hispanic compared with Hispanic examinees (OR, 2.4; 95% CI, 1.2-4.7), and single women compared with women who were married with children during internship (OR, 10.3; 95% CI, 2.1-51) were more likely to pass the certifying examination on the first try. Conclusions and Relevance:Resident race, ethnicity, sex, and family status at internship were observed to be associated with board passage rates. There are multiple possible explanations for these worrisome observations that need to be explored. Tracking demographics of trainees to help understand passage rates based on demographics will be important. The American Board of Surgery already has begun addressing the potential for unconscious bias among board examiners by increasing diversity and adding implicit bias training.

Project description:BackgroundExamination of arthroscopic skill requires evaluation tools that are valid and reliable with clear criteria for passing. The Arthroscopic Surgery Skill Evaluation Tool was developed as a video-based assessment of technical skill with criteria for passing established by a panel of experts. The purpose of this study was to test the validity and reliability of the Arthroscopic Surgery Skill Evaluation Tool as a pass-fail examination of arthroscopic skill.MethodsTwenty-eight residents and two sports medicine faculty members were recorded performing diagnostic knee arthroscopy on a left and right cadaveric specimen in our arthroscopic skills laboratory. Procedure videos were evaluated with use of the Arthroscopic Surgery Skill Evaluation Tool by two raters blind to subject identity. Subjects were considered to pass the Arthroscopic Surgery Skill Evaluation Tool when they attained scores of ≥ 3 on all eight assessment domains.ResultsThe raters agreed on a pass-fail rating for fifty-five of sixty videos rated with an interclass correlation coefficient value of 0.83. Ten of thirty participants were assigned passing scores by both raters for both diagnostic arthroscopies performed in the laboratory. Receiver operating characteristic analysis demonstrated that logging more than eighty arthroscopic cases or performing more than thirty-five arthroscopic knee cases was predictive of attaining a passing Arthroscopic Surgery Skill Evaluation Tool score on both procedures performed in the laboratory.ConclusionsThe Arthroscopic Surgery Skill Evaluation Tool is valid and reliable as a pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory.Clinical relevanceThis study demonstrates that the Arthroscopic Surgery Skill Evaluation Tool may be a useful tool for pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory. Further study is necessary to determine whether the Arthroscopic Surgery Skill Evaluation Tool can be used for the assessment of multiple arthroscopic procedures and whether it can be used to evaluate arthroscopic procedures performed in the operating room.

Project description:BackgroundDespite similar performance metrics, women medical trainees routinely self-assess their own skills lower than men. The phenomenon of a "confidence gap" between genders, where women report lower self-confidence independent of actual ability or competency, may have an important interaction with gender differences in assessment. Identifying whether there are gender-based differences in how confidence is mentioned in written evaluations is a necessary step to understand the interaction between evaluation and the gender-based confidence gap.ObjectiveTo analyze faculty evaluations of internal medicine (IM) residents for gender-based patterns in the use of iterations of "confidence."DesignWe performed a retrospective cohort study of all inpatient faculty evaluations of University of Pennsylvania IM residents from 2018 to 2021. We performed n-gram text-mining to identify evaluations containing the terms "confident," "confidence," or "confidently." We performed univariable and multivariable logistic regression to determine the association between resident gender and references to confidence (including comments reflecting too little confidence), adjusting for faculty gender, post-graduate year (PGY), numeric rating, and service.SubjectsUniversity of Pennsylvania IM residents from 2018 to 2021.Key resultsThere were 5416 evaluations of IM residents (165 women [51%], 156 men [49%]) submitted by 356 faculty members (149 women [51%]), of which 7.1 % (n=356) contained references to confidence. There was a significant positive association between the mention of confidence and women resident gender (OR 1.54, CI 1.23-1.92; p<0.001), which persisted after adjustment for faculty gender, numeric rating, and PGY level. Eighty evaluations of the cohort explicitly mentioned the resident having "too little confidence," which was also associated with women resident gender (OR 1.66, CI 1.05-2.62; p=0.031).ConclusionNarrative evaluations of women residents were more likely to contain references to confidence, after adjustment for numerical score, PGY level, and faculty gender, which may perpetuate the gender-based confidence gap, introduce bias, and ultimately impact professional identity development.

Project description:Traditional Korean medicine (TKM) emphasizes individualized diagnosis and treatment. This uniqueness makes AI modeling difficult due to limited data and implicit processes. Large language models (LLMs) have demonstrated impressive medical inference, even without advanced training in medical texts. This study assessed the capabilities of GPT-4 in TKM, using the Korean National Licensing Examination for Korean Medicine Doctors (K-NLEKMD) as a benchmark. The K-NLEKMD, administered by a national organization, encompasses 12 major subjects in TKM. GPT-4 answered 340 questions from the 2022 K-NLEKMD. We optimized prompts with Chinese-term annotation, English translation for questions and instruction, exam-optimized instruction, and self-consistency. GPT-4 with optimized prompts achieved 66.18% accuracy, surpassing both the examination's average pass mark of 60% and the 40% minimum for each subject. The gradual introduction of language-related prompts and prompting techniques enhanced the accuracy from 51.82% to its maximum accuracy. GPT-4 showed low accuracy in subjects including public health & medicine-related law, internal medicine (2), and acupuncture medicine which are highly localized in Korea and TKM. The model's accuracy was lower for questions requiring TKM-specialized knowledge than those that did not. It exhibited higher accuracy in diagnosis-based and recall-based questions than in intervention-based questions. A significant positive correlation was observed between the consistency and accuracy of GPT-4's responses. This study unveils both the potential and challenges of applying LLMs to TKM. These findings underline the potential of LLMs like GPT-4 in culturally adapted medicine, especially TKM, for tasks such as clinical assistance, medical education, and research. But they also point towards the necessity for the development of methods to mitigate cultural bias inherent in large language models and validate their efficacy in real-world clinical settings.

Dataset Information

Effects of gender in resident evaluations and certifying examination pass rates.

Background

Methods

Results

Publications

Effects of gender in resident evaluations and certifying examination pass rates.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets