Dataset Information

Assessment of Entrustable Professional Activities Using a Web-Based Simulation Platform During Transition to Emergency Medicine Residency: Mixed Methods Pilot Study.

ABSTRACT:

Background

The 13 core entrustable professional activities (EPAs) are key competency-based learning outcomes in the transition from undergraduate to graduate medical education in the United States. Five of these EPAs (EPA2: prioritizing differentials, EPA3: recommending and interpreting tests, EPA4: entering orders and prescriptions, EPA5: documenting clinical encounters, and EPA10: recognizing urgent and emergent conditions) are uniquely suited for web-based assessment.

Objective

In this pilot study, we created cases on a web-based simulation platform for the diagnostic assessment of these EPAs and examined the feasibility and acceptability of the platform.

Methods

Four simulation cases underwent 3 rounds of consensus panels and pilot testing. Incoming emergency medicine interns (N=15) completed all cases. A maximum of 4 "look for" statements, which encompassed specific EPAs, were generated for each participant: (1) performing harmful or missing actions, (2) narrowing differential or wrong final diagnosis, (3) errors in documentation, and (4) lack of recognition and stabilization of urgent diagnoses. Finally, we interviewed a sample of interns (n=5) and residency leadership (n=5) and analyzed the responses using thematic analysis.

Results

All participants had at least one missing critical action, and 40% (6/15) of the participants performed at least one harmful action across all 4 cases. The final diagnosis was not included in the differential diagnosis in more than half of the assessments (8/15, 54%). Other errors included selecting incorrect documentation passages (6/15, 40%) and indiscriminately applying oxygen (9/15, 60%). The interview themes included psychological safety of the interface, ability to assess learning, and fidelity of cases. The most valuable feature cited was the ability to place orders in a realistic electronic medical record interface.

Conclusions

This study demonstrates the feasibility and acceptability of a web-based platform for diagnostic assessment of specific EPAs. The approach rapidly identifies potential areas of concern for incoming interns using an asynchronous format, provides feedback in a manner appreciated by residency leadership, and informs individualized learning plans.

SUBMITTER: Peng CR

PROVIDER: S-EPMC8663509 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionMedical education is moving toward a competency-based framework with a focus on assessment using the Accreditation Council for Graduate Medical Education Milestones. Assessment of individual competencies through milestones can be challenging. While competencies describe characteristics of the person, the entrustable professional activities (EPAs) concept refers to work-related activities. EPAs would not replace the milestones but would be linked to them, integrating these frameworks. Many core specialties have already defined EPAs for resident trainees, but EPAs have not yet been created for emergency medicine (EM). This paper describes the development of milestone-linked EPAs for EM.MethodsTen EM educators from across North America formed a consensus working group to draft EM EPAs, using a modified Glaser state-of-the-art approach. A reactor panel with EPA experts from the United States, Canada and the Netherlands was created, and an iterative process with multiple revisions was performed based on reactor panel input. Following this, the EPAs were sent to the Council of Residency Directors for EM (CORD-EM) listserv for additional feedback.ResultsThe product was 11 core EPAs that every trainee from every EM program should be able to perform independently by the time of graduation. Each EPA has associated knowledge, skills, attitudes and behaviors (KSAB), which are either milestones themselves or KSABs linked to individual milestones. We recognize that individual programs may have additional focus areas or work-based activities they want their trainees to achieve by graduation; therefore, programs are also encouraged to create additional program-specific EPAs.ConclusionThis set of 11 core, EM-resident EPAs can be used as an assessment tool by EM residency programs, allowing supervising physicians to document the multiple entrustment decisions they are already making during clinical shifts with trainees. The KSAB list within each EPA could assist supervisors in giving specific, actionable feedback to trainees and allow trainees to use this list as an assessment-for-learning tool. Linking each KSAB to individual EM milestones allows EPAs to directly inform milestone assessment for clinical competency committees. These EPAs serve as another option for workplace-based assessment, and are linked to the milestones to create an integrated framework.

Project description:PurposeAs global health education and training shift toward competency-based approaches, academic institutions and organizations must define appropriate assessment strategies for use across health professions. The authors aim to develop entrustable professional activities (EPAs) for global health to apply across academic and workplace settings.MethodIn 2019, the authors invited 55 global health experts from medicine, nursing, pharmacy, and public health to participate in a multiround, online Delphi process; 30 (55%) agreed. Experts averaged 17 years of global health experience, and 12 (40%) were from low- to middle-income countries. In round one, participants listed essential global health activities. The authors used in vivo coding for round one responses to develop initial EPA statements. In subsequent rounds, participants used 5-point Likert-type scales to evaluate EPA statements for importance and relevance to global health across health professions. The authors elevated statements that were rated 4 (important/relevant to most) or 5 (very important/relevant to all) by a minimum of 70% of participants (decided a priori) to the final round, during which participants evaluated whether each statement represented an observable unit of work that could be assigned to a trainee. Descriptive statistics were used for quantitative data analysis. The authors used participant comments to categorize EPA statements into role domains.ResultsTwenty-two EPA statements reached at least 70% consensus. The authors categorized these into 5 role domains: partnership developer, capacity builder, data analyzer, equity advocate, and health promoter. Statements in the equity advocate and partnership developer domains had the highest agreement for importance and relevance. Several statements achieved 100% agreement as a unit of work but achieved lower levels of agreement regarding their observability.ConclusionsEPAs for global health may be useful to academic institutions and other organizations to guide the assessment of trainees within education and training programs across health professions.