Dataset Information

Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors.

ABSTRACT:

Background

Multi-source evaluation has demonstrated value for trainees, but is not generally provided to residency or fellowship program directors (PDs).

Objective

To develop, implement, and evaluate a PD multi-source evaluation process.

Methods

Tools were developed for PD evaluation by trainees, department chairs, and graduate medical education (GME) leadership. Evaluation questions were based on PD responsibilities, including Accreditation Council for Graduate Medical Education (ACGME) requirements. A follow-up survey assessed the process.

Results

Evaluation completion rates were as follows: trainees in academic year 2012-2013, 53% (958 of 1824), and in academic year 2013-2014, 42% (800 of 1898); GME directors in 2013-2014, 100% (95 of 95); and chairs/chiefs in 2013-2014, 92% (109 of 118). Results of a follow-up survey of PDs (66%, 59 of 90) and chairs (74%, 48 of 65) supports the evaluations' value, with 45% of responding PDs (25 of 56) and 50% of responding chairs (21 of 42) characterizing them as "extremely" or "quite" useful. Most indicated this was the first written evaluation they had received (PDs 78%, 46 of 59) or provided (chairs 69%, 33 of 48) regarding the PD role. More than 60% of PD (30 of 49) and chair respondents (24 of 40) indicated trainee feedback was "extremely" or "quite" useful, and nearly 50% of PDs (29 of 59) and 21% of chairs (10 of 48) planned changes based on the results. Trainee response rates improved in 2014-2015 (52%, 971 of 1872) and 2015-2016 (69%, 1276 of 1837).

Conclusions

In our institution, multi-source evaluation of PDs was sustained over 4 years with acceptable and improving evaluation completion rates. The process and assessment tools are potentially transferrable to other institutions.

SUBMITTER: Goldhamer ME

PROVIDER: S-EPMC5058595 | biostudies-literature | 2016 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors.

Goldhamer Mary Ellen J ME Baker Keith K Cohen Amy P AP Weinstein Debra F DF

Journal of graduate medical education 20161001 4

<h4>Background</h4>Multi-source evaluation has demonstrated value for trainees, but is not generally provided to residency or fellowship program directors (PDs).<h4>Objective</h4>To develop, implement, and evaluate a PD multi-source evaluation process.<h4>Methods</h4>Tools were developed for PD evaluation by trainees, department chairs, and graduate medical education (GME) leadership. Evaluation questions were based on PD responsibilities, including Accreditation Council for Graduate Medical Edu ...[more]

PMID: 27777673

Similar Datasets

Project description:BackgroundLittle is known about the level of burnout among program administrators (PAs) in medical education and its impact on the trainee environment.ObjectiveTo investigate variations in burnout levels over a 1-year period among a national cohort of PAs and examine any associations between perceived support and isolation.DesignA 1-year longitudinal study conducted to assess burnout levels among PAs across the USA. The Copenhagen Burnout Inventory (score range, 0-100) was used to measure burnout over one academic year (July 2017-June 2018). The generalized estimating equations model was used to measure changes in burnout levels from the start of the academic year. To explore the differences in burnout scores across question response levels, a one-way ANOVA test was utilized and reported as least squares means ± SD.ParticipantsIndividuals who self-identified as PAs in a graduate medical education training program. Among the 1084 persons nationwide who expressed interest, 904 (83%) completed the baseline survey; 29 of the 42 (69%) local administrators completed the survey. "Clients" defined as interns, residents/fellows, and medical students.Main measuresChange in burnout score using the validated tool. Hypothesis formulated prior to data collection.Key resultsAmong the 931 participants, the 3rd quarter (March 2018) marked the lowest average personal burnout score (change from the start of academic year, - 3.67; p < 0.001, 95% CI - 5.77 to - 1.58) and work-related burnout score (change, - 3.03; p < 0.001, 95% CI - 5.01 to - 1.06). Client-related burnout was the lowest in September 2017 (change, - 1.46; p = 0.491; 95% CI - 3.54 to 0.62). June 2018: those who strongly agreed to feeling isolated in their current position had an increased personal (69.1 ± 18.4 SD), work-related (72.5 ± 20.8 SD), and client-related (42.3 ± 23.7 SD) burnout score.ConclusionsPA burnout levels fluctuate over the academic year and are shown to increase as feelings of isolation grow.

Project description:ObjectivesOur program implemented East EMWars, a year-long, longitudinal game that added competition to our existing curricular content. We surveyed residents to investigate the impact of gamification in emergency medicine residency training. We hypothesized that residents would report higher levels of motivation, engagement, and challenge with gamification compared to traditional didactics. Furthermore, we hypothesized that residents would exhibit generally positive perceptions about gamification as a learning tool and that it would translate to improved performance on the annual in-training examination.MethodsThis was a single-center, prospective pre- and post-intervention survey study at a community-based emergency medicine residency program. Given the multiplicity of research questions and inherent nature of educational research, a mixed methods approach was utilized. We utilized nonparametric testing for quantitative data with paired responses pre- and post-intervention. We solicited comments on the post-intervention that were categorized under thematic approach and presented in complete and unedited form in the results.ResultsEighteen (100%) of eligible residents in our program participated in both surveys. There were statistically significant increases in reported levels of motivation, engagement, and challenge with gamification compared to traditional didactic methods. Residents also reported overwhelmingly positive general perceptions about gamification and its broader generalizability and applicability. We did not reach statistical significance in determining if in-training exam scores were associated with our gamification initiative.ConclusionsThis study was a first-of-its-kind look into a longitudinal game in an emergency medicine residency program. Although our results are encouraging, medical educators need further research to determine if this increase in motivation, engagement, and challenge will be associated with an increase in examination scores or, more importantly, healthcare outcomes. Theory-based, broader-scale, prospective studies are needed to further explore and help establish these associations and outcomes.

Project description:BACKGROUND: Little is known about the factors that influence applicants' decisions to select a graduate medical education training program. Programs may improve their recruitment by better understanding that process. METHODS: Following the Match for the years 2008 though 2010, an electronic survey was sent to 664 applicants to 15 Duke University graduate medical education programs. Those individuals were ranked high enough to match to Duke University but chose to match at another institution. We used quantitative and qualitative analyses to explore factors influencing candidates' decisions and to identify program and institutional improvements to better attract top candidates. RESULTS: Response rate was 63% (417/664). The quantitative analysis identified factors that were "very important" to applicants (current resident satisfaction, relationship between faculty and residents, collegiality of current residents, quantity and quality of faculty content/mentoring, faculty teaching expertise, diversity of patients and types of procedures, and location), and those that were "not important" to applicants (child care or education opportunities for children, salary, United States Medical Licensing Examination requirements, part-time training options, favorable environment for minorities or women). There were statistically significant differences among specialties. Qualitative analysis identified 3 themes as most influential in their choice: location, program relationships, and the interview experience. CONCLUSION: The collection of 3 years of data across specialties identified factors that are important to applicants' choice of a training program. Some factors can be addressed quickly and with little effort, such as the format of the interview day. Others require programs and institutions to revisit their mission, priorities, and training culture to improve relationships between residents and faculty. Programs may wish to be more tactical in selecting the residents and faculty who meet with applicants and be more focused in planning how the interview time is used.

Dataset Information

Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors.

Background

Objective

Methods

Results

Conclusions

Publications

Evaluating the Evaluators: Implementation of a Multi-Source Evaluation Program for Graduate Medical Education Program Directors.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets