Dataset Information

Learner perception of oral and written examinations in an international medical training program.

ABSTRACT:

Background

There are an increasing number of training programs in emergency medicine involving different countries or cultures. Many examination types, both oral and written, have been validated as useful assessment tools around the world; but learner perception of their use in the setting of cross-cultural training programs has not been described.

Aims

The goal of this study was to evaluate learner perception of four common examination methods in an international educational curriculum in emergency medicine.

Methods

Twenty-four physicians in a cross-cultural training program were surveyed to determine learner perception of four different examination methods: structured oral case simulations, multiple-choice tests, semi-structured oral examinations, and essay tests. We also describe techniques used and barriers faced.

Results

There was a 100% response rate. Learners reported that all testing methods were useful in measuring knowledge and clinical ability and should be used for accreditation and future training programs. They rated oral examinations as significantly more useful than written in measuring clinical abilities (p < 0.01). Compared to the other three types of examinations, learners ranked oral case simulations as the most useful examination method for assessing learners' fund of knowledge and clinical ability (p < 0.01).

Conclusions

Physician learners in a cross-cultural, international training program perceive all four written and oral examination methods as useful, but rate structured oral case simulations as the most useful method for assessing fund of knowledge and clinical ability.

SUBMITTER: Kelly SP

PROVIDER: S-EPMC2850976 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Objective: The final year of undergraduate medical education in Germany is called the practical year (PY), where emphasis is placed on developing practical skills requisite of soon-to-be physicians. Many students choose to spend part of this year abroad, yet little is known about their results on the medical licensing examinations. Is there a predisposition of high-performing students to go abroad as compared to lower performers? Are international health electives during the PY followed by higher scores in the final section of the German medical licensing examination (GMLE2)? Methods: We conducted a retrospective cohort study among undergraduate medical students at the LMU Munich, who participated in the final section of the German medical licensing examination between autumn 2009 and spring 2011. Of the 1,731 eligible students, 554 (32%) participated in our study. We analyzed for statistical associations of international health electives with written test scores of both sections of the medical licensing examination as well as grades earned during medical school training. We then used multiple regression analysis to identify relevant predictors of GMLE2 scores. Results: Approximately half of study participants pursued international health electives during the PY (51.1%). The number of students going abroad increased with the scores achieved on the first section of the medical licensing examination (GMLE1, p<0.001). Stratified by their GMLE1 scores students who pursued electives abroad during their PY achieved higher GMLE2 scores (p<0.001). The strongest predictor for GMLE2 scores were grades obtained during medical school training; age and study duration indicated lower scores; and those engaging in international health electives correlated with higher scores. Conclusions: Students with higher GMLE1 scores go abroad during PY more often. Beyond that, students who pursue international health electives achieve higher GMLE2 scores than those who stay in Germany during PY. There is an unmet need for additional research to identify which factors make these students perform better and what motivates them to go abroad.

Project description:ProblemWritten feedback is often overly positive, nonspecific, and difficult to interpret. Learner satisfaction with written feedback is low and obtaining written feedback that encourages self-reflection is challenging. Improving feedback quality is laborious and only modestly effective.ApproachThe authors developed the LEAF (Learner-Engaged Analysis of Feedback) method to improve learner satisfaction with, and reflection on, existing written feedback. The method pairs a learner and coach to methodically identify themes in the learner's written feedback. Themes occurring more frequently or less frequently than typical offer areas for reflection, as they may identify learners' relative strengths or weaknesses. The method was introduced at the Massachusetts General Hospital in 2017 during program director (PD) meetings with anesthesiology residents. In 2018, resident satisfaction was measured (1 to 5 Likert-type questions, 1 = "not at all satisfied," 5 = "extremely satisfied") for 4 feedback sources, 2 related to the LEAF method (PD meetings, written feedback) and 2 unrelated (verbal feedback, mentor feedback). Residents' comments were qualitatively assessed to explore the impact on self-reflection.OutcomesResidents who had participated in a LEAF session (n = 54), compared with those who had not (n = 11), reported higher satisfaction with written feedback (mean 3.1 versus 2.5, d = 0.53, P = .03) and PD meeting feedback (mean 3.8 versus 2.8, d = 0.80, P = .03). There were no significant differences between groups for satisfaction with feedback unrelated to the LEAF method. Qualitative analysis of comments suggested that residents found the method useful for providing holistic self-assessment, facilitating goal setting, uncovering blind spots, and improving feedback interpretation.Next stepsNext steps should include studies determining if the association between increased learner satisfaction with written feedback and the LEAF method is causal, and whether this feedback process changes learners' subsequent behaviors.

Project description:BackgroundPakistan has an underdeveloped and overburdened emergency care system, with most emergency departments (EDs) staffed by physicians not formally trained in emergency medicine (EM). As of January 2020, only nine Pakistani institutions were providing formal EM specialty training; therefore, a training program of shorter duration is needed in the interim.MethodsThe Certification Program in Emergency Medicine (CPEM) is a 1-year training program in EM consisting of two arms: CPEM-Clinical (CPEM-C), which includes physicians from The Indus Hospital (TIH) ED, and CPEM-Didactic (CPEM-D), including physicians from EDs across Karachi. Both groups participate in weekly conferences, including didactics, small-group discussions, workshops, and journal clubs. CPEM-C learners also receive clinical mentorship from visiting international and TIH EM faculty. Both groups were assessed with preprogram, midterm, and final examinations as well as on clinical skills. Additionally, both groups provided regular feedback on program content and administration.ResultsTwenty-five of the 32 initially enrolled learners completed the program in June 2019. Scores on a matched set of 50 questions administered in the pretest and final examination improved by an average of 15.1% (p < 0.005) for CPEM-C learners and 8.5% (p < 0.0005) for CPEM-D learners, with 93% of learners showing improvement. Clinical evaluations of CPEM-C and CPEM-D learners during the first and fourth quarters showed an average improvement of 1.1 out of 5 (p < 0.05) and 1.2 out of 9 (p < 0.0005) points, respectively. Learner evaluations of the program were overall positive.ConclusionsCPEM demonstrated significant improvement in test scores and clinical evaluations in both program arms. Evaluations also suggested that the program was well received. These data, along with CPEM's ability to train physicians from multiple institutions using low-cost, innovative educational strategies, suggest that it may be an effective, transferable mechanism for the expedited development of EM in Pakistan and countries where EM is developing as a specialty.