Dataset Information

E-ASSESS: Creating an EPA Assessment Tool for Structured Simulated Emergency Scenarios.

ABSTRACT:

Background

The entrustable professional activity (EPA) assessment framework allows supervisors to assign entrustment levels to physician trainees for specific activities. Limited opportunity for direct observation of trainees hampers entrustment decisions, in particular for infrequently performed activities. Simulation allows for direct observation, so tools to assess performance of EPAs in simulation could potentially provide additional data to complement clinical assessments.

Objective

We developed and collected validity evidence for a simulation-based tool grounded in the EPA framework.

Methods

We developed E-ASSESS (EPA Assessment for Structured Simulated Emergency ScenarioS) to assess performance in 2 EPAs among pediatric residents participating in simulation-based team training in 2017-2018. We collected validity data, applying Messick's unitary view. Three raters used E-ASSESS to assign entrustment levels based on performance in simulation. We compared those ratings to entrustment levels assigned by clinical supervisors (different from the study raters) for the same residents on a separate tool designed for clinical practice. We calculated intraclass correlation (ICC) for each tool and Pearson correlation coefficients to compare ratings between tools.

Results

Twenty-eight residents participated in the study. The ICC between the 3 raters for entrustment ratings on E-ASSESS ranged from 0.65 to 0.77, while ICC among raters of the clinical tool were 0.59 and 0.57. We found no significant correlations between E-ASSESS ratings and clinical practice ratings for either EPA (r = -0.35 and 0.38, P > .05).

Conclusions

Assessment following an EPA framework in the simulation context may be useful to provide data points to inform entrustment decisions as part of resident assessment.

SUBMITTER: Andler C

PROVIDER: S-EPMC7161329 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:While metapopulation theory offers tractable means to understand extinction risks for patchily-distributed endangered species, real systems often feature discrepant patch quality and accessibility, complex influences of environmental stochasticity, and regional and temporal autocorrelation. Spatially structured metapopulation models are flexible and can use real data but often at the cost of generality. Particularly as resources for management of such species are often critically limited, endangered species management guided by metapopulation modeling requires incorporation of biological realism. Here we developed a flexible, stochastic spatially structured metapopulation model of the profoundly endangered Amargosa vole, a microtine rodent with an extant population of only a few hundred individuals within 1km2 of habitat in the Mojave Desert. Drought and water insecurity are increasing extinction risks considerably. We modelled subpopulation demographics using a Ricker-like model with migration implemented in an incidence function metapopulation model. A set of scenarios was used to assess the effect of anthropogenic stressors or management actions on expected time to extinction (Te) including: 1) wildland fire, 2) anthropogenically-mediated losses of hydrologic flows, 3) drought, 4) intentional expansion of existing patches into 'megamarshes' (i.e. via restoration/enhancement), and 5) additive impacts of multiple influences. In isolation, marshes could be sources or sinks, but spatial context within the full metapopulation including adjacency could alter relative impacts of subpopulations on all other subpopulations. The greatest reductions in persistence occurred in scenarios simulated with impacts from drought in combination with fire or anthropogenically-mediated losses of hydrologic flows. Optimal actions to improve persistence were to prevent distant and smaller marshes from acting as sinks through strategic creation of megamarshes that act as sources of voles and stepping-stones. This research reinforces that management resources expended without guidance from empirically-based modeling can actually harm species' persistence. This metapopulation-PVA tool could easily be implemented for other patchily-distributed endangered species and allow managers to maximize scarce resources to improve the likelihood of endangered species persistence.

Project description:ObjectivesIn 2016, a national consensus conference created the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM), a standardized end-of-shift assessment tool. We report the first large-scale analysis of professionalism concerns collected from May 2017 through December 2018 by a multisite consortium using the NCAT-EM. Our primary objective was to characterize the nature and frequency of professionalism concerns. Our secondary objective was to identify characteristics associated with giving or receiving a professionalism flag.MethodsThe consortium database includes assessments for all students on EM clerkships at participating sites. This report presents descriptive statistics about the frequency of different flags, the distribution of flags among different student categories, assessor and student characteristics, and distribution of global assessment scores on assessments citing concerns. We used Fisher's exact test to look for associations between the frequency of professionalism flags and the sex of the students and assessors and across student categories. We used logistic regression to look for relationships between professionalism concerns and global assessment scores as well as intent to apply in EM.ResultsWe screened 6,768 assessments of 784 students by 719 assessors from 13 sites. After excluding assessments without flags and assessments with apparent data entry errors, we analyzed 57 (0.8%) assessments containing 79 flags. The most frequent flags were punctuality (25/79, 31.6%) and initiative (20/79, 25.3%). Few students received flags (42/784, 5.4%). Few assessors flagged concerns (41/719, 5.7%). We detected no correlation between the frequency of flags and whether a student was applying in EM or between the sex of students and assessors. Global scores of lower one-third appeared more often in assessments with a flag (30/57, 52.6% vs. 233/6,711, 3.5%).ConclusionsOnly 5.4% of students received flags. Punctuality and initiative accounted for a majority of citations. Professionalism flags correlated strongly with lower global assessment scores.