Dataset Information

Assessing Entrustable Professional Activities Using an Orientation OSCE: Identifying the Gaps.

ABSTRACT:

Background

A residency program's intern cohort is comprised of individuals from different medical schools that place varying levels of emphasis on Core Entrustable Professional Activities for Entering Residency (CEPAERs). Program directors have expressed concerns about the preparedness of medical school graduates. Though guiding principles for implementation of the CEPAERs have been published, studies using this framework to assess interns' baseline skills during orientation are limited.

Objective

A CEPAER-based objective structured clinical examination (OSCE) was implemented with the aims to (1) assess each intern's baseline clinical skills and provide formative feedback; (2) determine an intern's readiness for resident responsibilities; (3) inform individualized education plans; and (4) address identified gaps through curricular change.

Methods

During orientation, all 33 interns from internal medicine (categorical, preliminary, and medicine-psychiatry) participated in the OSCE. Six 20-minute stations evaluated 8 EPAs. Faculty completed a global assessment, and standardized patients completed a communications checklist and global assessment. All interns completed a self-assessment of baseline skills and a post-OSCE survey.

Results

Stations assessing handoffs, informed consent, and subjective, objective, assessment, and plan (SOAP) note were the lowest-performing stations. Interns performed lower in skills for which they did not report previous training. Formal instruction was incorporated into didactic sessions for the lowest-performing stations. The majority of interns indicated that the assessment was useful, and immediate feedback was beneficial.

Conclusions

This OSCE during orientation offers just-in-time baseline information regarding interns' critical skills and may lead to individualized feedback as well as continuous curricular improvement.

SUBMITTER: CarlLee S

PROVIDER: S-EPMC6476083 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Importance:Entrustable professional activities (EPAs) are an emerging workplace-based, patient-oriented assessment approach with limited empirical evidence. Objective:To measure the development of pediatric trainees' clinical skills over time using EPA-based assessment data. Design, Setting, and Participants:Prospective cohort study of categorical pediatric residents over 3 academic years (2015-2016, 2016-2017, and 2017-2018) assessed on 17 American Board of Pediatrics EPAs. Residents in training at 23 pediatric residency programs in the Association of Pediatric Program Directors Longitudinal Educational Assessment Research Network were included. Assessment was conducted by clinical competency committee members, who made summative assessment decisions regarding levels of supervision required for each resident and each EPA. Data were collected from May 2016 to November 2018 and analyzed from November to December 2018. Interventions:Longitudinal, prospective assessment using EPAs. Main Outcomes and Measures:Trajectories of supervision levels by EPA during residency training and how often graduating residents were deemed ready for unsupervised practice in each EPA. Results:Across the 5 data collection cycles, 1987 residents from all 3 postgraduate years in 23 residency programs were assigned 25 503 supervision level reports for the 17 general pediatrics EPAs. The 4 EPAs that required the most supervision across training were EPA 14 (quality improvement) on the 5-level scale (estimated mean level at graduation, 3.7; 95% CI, 3.6-3.7) and EPAs 8 (transition to adult care; mean, 7.0; 95% CI, 7.0-7.1), 9 (behavioral and mental health; mean, 6.6; 95% CI, 6.5-6.6), and 10 (resuscitate and stabilize; mean, 6.9; 95% CI, 6.8-7.0) on the expanded 5-level scale. At the time of graduation (36 months), the percentage of trainees who were rated at a supervision level corresponding to "unsupervised practice" varied by EPA from 53% to 98%. If performance standards were set to align with 90% of trainees achieving the level of unsupervised practice, this standard would be met for only 8 of the 17 EPAs (although 89% met this standard for EPA 17, performing the common procedures of the general pediatrician). Conclusions and Relevance:This study presents initial evidence for empirically derived practice readiness and sets the stage for identifying curricular gaps that contribute to discrepancy between observed practice readiness and standards needed to produce physicians able to meet the health needs of the patient populations they serve. Future work should compare these findings with postgraduation outcomes data as a means of seeking validity evidence.

Project description:PurposeAs global health education and training shift toward competency-based approaches, academic institutions and organizations must define appropriate assessment strategies for use across health professions. The authors aim to develop entrustable professional activities (EPAs) for global health to apply across academic and workplace settings.MethodIn 2019, the authors invited 55 global health experts from medicine, nursing, pharmacy, and public health to participate in a multiround, online Delphi process; 30 (55%) agreed. Experts averaged 17 years of global health experience, and 12 (40%) were from low- to middle-income countries. In round one, participants listed essential global health activities. The authors used in vivo coding for round one responses to develop initial EPA statements. In subsequent rounds, participants used 5-point Likert-type scales to evaluate EPA statements for importance and relevance to global health across health professions. The authors elevated statements that were rated 4 (important/relevant to most) or 5 (very important/relevant to all) by a minimum of 70% of participants (decided a priori) to the final round, during which participants evaluated whether each statement represented an observable unit of work that could be assigned to a trainee. Descriptive statistics were used for quantitative data analysis. The authors used participant comments to categorize EPA statements into role domains.ResultsTwenty-two EPA statements reached at least 70% consensus. The authors categorized these into 5 role domains: partnership developer, capacity builder, data analyzer, equity advocate, and health promoter. Statements in the equity advocate and partnership developer domains had the highest agreement for importance and relevance. Several statements achieved 100% agreement as a unit of work but achieved lower levels of agreement regarding their observability.ConclusionsEPAs for global health may be useful to academic institutions and other organizations to guide the assessment of trainees within education and training programs across health professions.