A Competency-based Tool for Resident Evaluation of Pediatric Emergency Department Faculty.
Ontology highlight
ABSTRACT:
SUBMITTER: Sandler E
PROVIDER: S-EPMC9897249 | biostudies-literature | 2022 Dec
REPOSITORIES: biostudies-literature
Publications
A Competency-based Tool for Resident Evaluation of Pediatric Emergency Department Faculty.
Sandler Ethan E Lee Michael M Carter Rebecca R Davis Natalie L NL Dubbs Sarah B SB Webster Karen K Donohue Katelyn E KE
The western journal of emergency medicine 20221221 1
Similar Datasets
Project description:IntroductionPediatric emergency medicine physicians struggle to maintain their critical procedural and resuscitation skills. Continuing professional development programs incorporating simulation and competency-based standards may help ensure skill maintenance. Using a logic model framework, we sought to evaluate the effectiveness of a mandatory annual competency-based medical education (CBME) simulation program.MethodsThe CBME program, evaluated from 2016 to 2018, targeted procedural, point-of-care ultrasound (POCUS) and resuscitation skills. Delivery of educational content included a flipped-classroom website, deliberate practice, mastery-based learning, and stop-pause debriefing. Participants' competence was assessed using a 5-point global rating scale (GRS; 3 = competent, 5 = mastery). Statistical process control charts were used to measure the effect of the CBME program on team performance during in situ simulations (ISS), measured using the Team Emergency Assessment Measure (TEAM) scale. Faculty completed an online program evaluation survey.ResultsForty physicians and 48 registered nurses completed at least one course over 3 years (physician mean ± SD 2.2 ± 0.92). Physicians achieved competence on 430 of 442 stations (97.3%). Mean ± SD GRS scores for procedural, POCUS, and resuscitation stations were 4.34 ± 0.43, 3.96 ± 0.35, and 4.17 ± 0.27, respectively. ISS TEAM scores for "followed standards and guidelines" improved significantly. No signals of special cause variation emerged for the other 11 TEAM items, indicating skills maintenance. Physicians rated CBME training as highly valuable (mean question scores 4.15-4.85/5). Time commitment and scheduling were identified as barriers to participation.ConclusionsOur mandatory simulation-based CBME program had high completion rates and very low station failures. The program was highly rated and faculty improved or maintained their ISS performance across TEAM scale domains.
| S-EPMC10033845 | biostudies-literature
Project description:BackgroundMaintaining acute care physician competence is critically important. Current maintenance of certification (MOC) programs has started to incorporate simulation-based education (SBE). However, competency expectations have not been defined. This article describes the development of a mandatory annual SBE, competency-based simulation program for technical and resuscitation skills for pediatric emergency medicine (PEM) physicians.MethodsThe competency-based medical education (CBME) program was introduced in 2016. Procedural skill requirements were based on a needs assessment derived from Royal College PEM training guidelines. Resuscitation scenarios were modified versions of pre-existing in-situ mock codes or critical incident cases. All full-time faculty were required to participate annually in both sessions. Delivery of educational content included a flipped classroom website, deliberate practice, and stop-pause debriefing. All stations required competency checklists and global rating scales.ResultsBetween 2016 and 2018, 40 physicians and 48 registered nurses attended these courses. Overall course evaluations in 2018 were 4.92/5 and 4.93/5. Barriers to implementation include the need for many simulation education experts, time commitment, and clinical scheduling during course events.ConclusionWe have developed a mandatory simulation-based, technical, and resuscitation CBME program for PEM faculty. This simulation-based CBME program could be adapted to other acute care disciplines. Further research is required to determine if these skills are enhanced both in a simulated and real environment and if there is an impact on patient outcomes.
| S-EPMC8101101 | biostudies-literature
Project description:ObjectivesThe outcome of emergency medicine (EM) training is to produce physicians who can competently run an emergency department (ED) shift. However, there are few tools with supporting validity evidence specifically designed to assess multiple key competencies across an entire shift. The investigators developed and gathered validity evidence for a novel entrustment-based tool to assess a resident's ability to safely run an ED shift.MethodsThrough a nominal group technique, local and national stakeholders identified dimensions of performance that are reflective of a competent ED physician and are required to safely manage an ED shift. These were included as items in the Ottawa Emergency Department Shift Observation Tool (O-EDShOT), and each item was scored using an entrustment-based rating scale. The tool was implemented in 2018 at the University of Ottawa Department of Emergency Medicine, and quantitative data and qualitative feedback were collected over 6 months.ResultsA total of 1,141 forms were completed by 78 physicians for 45 residents. An analysis of variance demonstrated an effect of training level with statistically significant increases in mean O-EDShOT scores with each subsequent postgraduate year (p < 0.001). Scores did not vary by ED treatment area. Residents rated as able to safely run the shift had significantly higher mean ± SD scores (4.8 ± 0.3) than those rated as not able (3.8 ± 0.6; p < 0.001). Faculty and residents reported that the tool was feasible to use and facilitated actionable feedback aimed at progression toward independent practice.ConclusionsThe O-EDShOT successfully discriminated between trainees of different levels regardless of ED treatment area. Multiple sources of validity evidence support the O-EDShOT as a tool to assess a resident's ability to safely run an ED shift. It can serve as a stimulus for daily observation and feedback making it practical to use within an EM residency program.
| S-EPMC7592826 | biostudies-literature
Project description:ObjectivesProvider efficiency has been reported in the literature but there is a lack of efficiency analysis among emergency medicine (EM) residents. We aim to compare efficiency of EM residents of different training levels and determine if EM resident efficiency is affected by emergency department (ED) crowding.MethodsWe conducted a single-center retrospective observation study from July 1, 2014, to June 30, 2017. The number of new patients per resident per hour and provider-to-disposition (PTD) time of each patient were used as resident efficiency markers. A crowding score was assigned to each patient upon the patient's arrival to the ED. We compared efficiency among EM residents of different training levels under different ED crowding statuses. Dynamic efficiency changes were compared monthly through the entire academic year (July to next June).ResultsThe study enrolled a total of 150,920 patients. A mean of 1.9 patients/hour was seen by PGY-1 EM residents in comparison to 2.6 patients/hour by PGY-2 and -3 EM residents. Median PTD was 2.8 hours in PGY-1 EM residents versus 2.6 hours in PGY-2 and -3 EM residents. There were no significant differences in acuity across all patients seen by EM residents. When crowded conditions existed, residency efficiency increased, but such changes were minimized when the ED became overcrowded. A linear increase of resident efficiency was observed only in PGY-1 EM residents throughout the entire academic year.ConclusionResident efficiency improved significantly only during their first year of EM training. This efficiency can be affected by ED crowding.
| S-EPMC6637007 | biostudies-literature
Project description:Effective real-time feedback is critical to medical education. This study tested the hypothesis that an educational intervention related to feedback would improve emergency medicine (EM) faculty and resident physician satisfaction with feedback.This was a cluster-randomized, controlled study of 15 EM residency programs in 2007-2008. An educational intervention was created that combined a feedback curriculum with a card system designed to promote timely, effective feedback. Sites were randomized either to receive the intervention or to continue their current feedback method. All participants completed a Web-based survey before and after the intervention period. The primary outcome was overall feedback satisfaction on a 10-point scale. Additional items addressed specific aspects of feedback. Responses were compared using a generalized estimating equations model, adjusting for confounders and baseline differences between groups. The study was designed to achieve at least 80% power to detect a one-point difference in overall satisfaction (? = 0.05).Response rates for pre- and postintervention surveys were 65.9 and 47.3% (faculty) and 64.7 and 56.9% (residents). Residents in the intervention group reported a mean overall increase in feedback satisfaction scores compared to those in the control group (mean increase 0.96 points, standard error [SE] ± 0.44, p = 0.03) and significantly higher satisfaction with the quality, amount, and timeliness of feedback. There were no significant differences in mean scores for overall and specific aspects of satisfaction between the faculty physician intervention and control groups.An intervention designed to improve real-time feedback in the ED resulted in higher resident satisfaction with feedback received, but did not affect faculty satisfaction with the feedback given.
| S-EPMC3095955 | biostudies-literature
Project description:Pediatric residents report a lack of confidence and competence with procedural skills at graduation. Training programs could benefit from improved approaches to target these needs. Using the Institute for Healthcare Improvement (IHI) Model for Improvement and three Plan-Do-Study-Act (PDSA) cycles, we examined the impact of a procedure simulation boot camp on self-reported procedural confidence and competence as well as the longitudinal impacts of these sequential interventions on Accreditation Council for Graduate Medical Education (ACGME) Graduating Resident Survey (GRS) results.Three rapid cycle interventions were performed in successive academic years. The interventions included 1) increased awareness of available procedural experiences, 2) institution of procedural educational conferences, and 3) implementation of a senior resident procedure boot camp. Senior resident self-reported procedural confidence was measured before and after the boot camp. Procedural competence was measured using the ACGME GRS. Results: Thirty-two of 34 senior residents (94%) completed the 2016 ACGME GRS, similar to the response rates of 2014 (92%) and 2015 (94%), and 30 of 34 third-year residents participated in the procedure boot camp (88%). Resident confidence and competence with procedural skills improved after the institution of the quality improvement intervention. ACGME GRS-reported competency increased in bag and mask ventilation (77% to 94%), neonatal endotracheal intubation (39% to 69%), peripheral IV placement (10% to 50%), and umbilical catheter placement (35% to 53%).A quality improvement intervention with three rapid PDSA cycles was successful in improving senior pediatric resident confidence and competence with ACGME required procedural skills.
| S-EPMC5493469 | biostudies-other
Project description:Background:Assessment of trainees' competency is challenging; the predictive power of traditional evaluations is debatable especially in regard to noncognitive traits. New assessments need to be sought to better understand affective areas like personality. Grit, defined as "perseverance and passion for long-term goals," can assess aspects of personality. Grit predicts educational attainment and burnout rates in other populations and is accurate with an informant report version. Self-assessments, while useful, have inherent limitations. Faculty's ability to accurately assess trainees' grit could prove helpful in identifying learner needs and avenues for further development. Objective:This study sought to determine the correlation between EM resident self-assessed and faculty-assessed Grit Scale (Grit-S) scores of that same resident. Methods:Subjects were PGY-1 to -4 EM residents and resident-selected faculty as part of a larger multicenter trial involving 10 EM residencies during 2017. The Grit-S Scale was administered to participating EM residents; an informant version was completed by their self-selected faculty. Correlation coefficients were computed to assess the relationship between residents' self-assessed and the residents' faculty-assessed Grit-S score. Results:A total of 281 of 303 residents completed the Grit-S, for a 93% response rate; 200 of 281 residents had at least one faculty-assessed Grit-S score. No correlation was found between residents' self-assessed and faculty-assessed Grit-S scores. There was a correlation between the two faculty-assessed Grit-S scores for the same resident. Conclusion:There was no correlation between resident and faculty-assessed Grit-S scores; additionally, faculty-assessed Grit-S scores of residents were higher. This corroborates the challenges faculty face at accurately assessing aspects of residents they supervise. While faculty and resident Grit-S scores did not show significant concordance, grit may still be a useful predictive personality trait that could help shape future training.
| S-EPMC6339547 | biostudies-literature
Project description: Not available
| S-EPMC10715784 | biostudies-literature
Project description:IntroductionTraditional normative Likert-type evaluations of faculty teaching have several drawbacks, including lack of granular feedback, potential for inflation, and the halo effect. To provide more meaningful data to faculty on their teaching skills and encourage educator self-reflection and skill development, we designed and implemented a milestone-based faculty clinical teaching evaluation tool.MethodsThe evaluation tool contains 10 questions that assess clinical teaching skills with descriptive milestone behavior anchors. Nine of these items are based on the Stanford Faculty Development Clinical Teaching Model and annual Accreditation Council for Graduate Medical Education (ACGME) resident survey questions; the tenth was developed to address professionalism at our institution. The tool was developed with input from residency program leaders, residents, and the faculty development committee and piloted with graduate medical education learners before implementation.ResultsMore than 7,200 faculty evaluations by learners and 550 faculty self-evaluations have been collected. Learners found the form easy to use and preferred it to previous Likert-based evaluations. Over the 2 years that faculty self-evaluations have been collected, their scores have been similar to the learner evaluation scores. The feedback provided faculty with more meaningful data on teaching skills and opportunities for reflection and skill improvement and was used in constructing faculty teaching skills programs at the institutional level.DiscussionThis innovation provides an opportunity to give faculty members more meaningful teaching evaluations and feedback. It should be easy for other institutions and programs to implement. It leverages a familiar milestone construct and incorporates important ACGME annual resident survey information.
| S-EPMC6374742 | biostudies-literature
Project description:BackgroundDuring the coronavirus disease 2019 (COVID-19) pandemic period, the use of emergency services with pediatric non-COVID patients has decreased considerably. We aimed to examine whether there was a change in the demographic data, triage profile, causes, management, and cost of pediatric emergency department (PED) visits of non-COVID patients during the pandemic period.MethodsThis study was a retrospective, single-center, observational comparative study that was conducted at the PED. Patient records were examined during "the pandemic spring" and the same period of the previous year. Patient demographics, waiting time, and outcome of the PED visit were analyzed in the entire population of children admitted to the PED during the study period, whereas more precise data such as the reason for PED use, duration of symptoms, urgency levels according to the Emergency Severity Index (ESI), final diagnosis, management, and cost of patient care were analyzed in a sample of admitted patients. We used the chi-square test, Fisher's exact test, and Mann-Whitney U test for statistical analyses.ResultsA total of 62,593 PED visits occurred. During the pandemic period, PED visits showed a decrease of 55.8% compared to the previous year. Patients included in the sampling study group were selected using a systematic random sampling method. The median waiting time during the pandemic period was significantly shorter than the previous year (median 14 min [IQR: 5-32] vs. median 5 min [IQR: 2-16]; p<0.001). The median duration of symptoms was 1 day (1-2) in both groups. Emergency Severity Index (ESI) levels I, II, and III showed a significant increase (27.7% vs. 37.3%) in triage scoring compared to levels IV and V (72.3% vs. 62.7%) during the pandemic period (p<0.001). The median cost per patient during the pandemic period was statistically higher compared to the previous year ($19.57 [19.57-40.50] vs. $25.34 [31.50-52.01]; p<0.001). Overall costs during the pandemic period had a 1.6-fold decline.ConclusionWe highlighted the changes in an ordinary PED profile during an extraordinary period. A shift in ESI levels in a more emergent direction was observed. While the number of nonurgent patients, especially those with infections, decreased, the rates of surgical cases, acute neurological and heart diseases, home accidents, and poisoning increased relative to the pre-pandemic period.
| S-EPMC8492613 | biostudies-literature