Dataset Information

Seeing potential opportunities for teaching (SPOT): Evaluating a bundle of interventions to augment entrustable professional activity acquisition.

ABSTRACT:

Introduction

Within the Canadian competency-based medical education system, entrustable professional activities (EPAs) are used to assess residents on performed clinical duties. This study aimed to determine whether implementing a bundle of two interventions (a case-based discussion intervention and a rotation-based nudging system) could increase the number of EPA assessments that could occur for our trainees.

Methods

The authors designed an intervention bundle with two components: 1) a case-based workshop where trainees discussed which EPAs could be assessed with multiple cases and 2) a nudging system wherein each trainee was reminded of EPAs that would be useful to them on each rotation in their first year. We conducted a retrospective program evaluation to compare the intervention cohort (2019) to two historical cohorts using similar EPAs (2017, 2018).

Results

Data from 22 trainees (seven in 2017, eight in 2018, and seven in 2019) were analyzed. There was a marked increase in the total number of EPA assessments acquired in the 2019 cohort (average per resident = 285.7, 95% confidence interval [CI] = 256.1 to 312.3, range = 195-350) compared to the two other years (2018 [average = 132.4, 95% CI = 107.5 to 157.02, range = 107-167] and 2017 [70.1, 95% CI 45.3 to 91.0, range = 49-95]), yielding an effect size of Cohen's d = 4.02 for our intervention bundle.

Conclusions

Within the limitations of a small sample size, there was a strong effect of introducing two interventions (a case-based orientation and a nudging system) upon EPA assessments with PGY-1 residents. These strategies may be useful to others seeking to improve EPA assessment numbers in other specialties and clinical environments.

SUBMITTER: Sample S

PROVIDER: S-EPMC8381386 | biostudies-literature | 2021 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Seeing potential opportunities for teaching (SPOT): Evaluating a bundle of interventions to augment entrustable professional activity acquisition.

Sample Spencer S Al Rimawi Hussein H Bérczi Beatrix B Chorley Alexander A Pardhan Alim A Chan Teresa M TM

AEM education and training 20210801 4

<h4>Introduction</h4>Within the Canadian competency-based medical education system, entrustable professional activities (EPAs) are used to assess residents on performed clinical duties. This study aimed to determine whether implementing a bundle of two interventions (a case-based discussion intervention and a rotation-based nudging system) could increase the number of EPA assessments that could occur for our trainees.<h4>Methods</h4>The authors designed an intervention bundle with two components ...[more]

PMID: 34471797

Similar Datasets

Project description:PurposeAs global health education and training shift toward competency-based approaches, academic institutions and organizations must define appropriate assessment strategies for use across health professions. The authors aim to develop entrustable professional activities (EPAs) for global health to apply across academic and workplace settings.MethodIn 2019, the authors invited 55 global health experts from medicine, nursing, pharmacy, and public health to participate in a multiround, online Delphi process; 30 (55%) agreed. Experts averaged 17 years of global health experience, and 12 (40%) were from low- to middle-income countries. In round one, participants listed essential global health activities. The authors used in vivo coding for round one responses to develop initial EPA statements. In subsequent rounds, participants used 5-point Likert-type scales to evaluate EPA statements for importance and relevance to global health across health professions. The authors elevated statements that were rated 4 (important/relevant to most) or 5 (very important/relevant to all) by a minimum of 70% of participants (decided a priori) to the final round, during which participants evaluated whether each statement represented an observable unit of work that could be assigned to a trainee. Descriptive statistics were used for quantitative data analysis. The authors used participant comments to categorize EPA statements into role domains.ResultsTwenty-two EPA statements reached at least 70% consensus. The authors categorized these into 5 role domains: partnership developer, capacity builder, data analyzer, equity advocate, and health promoter. Statements in the equity advocate and partnership developer domains had the highest agreement for importance and relevance. Several statements achieved 100% agreement as a unit of work but achieved lower levels of agreement regarding their observability.ConclusionsEPAs for global health may be useful to academic institutions and other organizations to guide the assessment of trainees within education and training programs across health professions.

Project description:Importance:Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective:To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants:Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions:Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures:Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results:Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance:This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.

Dataset Information

Seeing potential opportunities for teaching (SPOT): Evaluating a bundle of interventions to augment entrustable professional activity acquisition.

Introduction

Methods

Results

Conclusions

Publications

Seeing potential opportunities for teaching (SPOT): Evaluating a bundle of interventions to augment entrustable professional activity acquisition.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets