Dataset Information

Feedback With Performance Metric Scorecards Improves Resident Satisfaction but Does Not Impact Clinical Performance.

ABSTRACT: Objectives:The Emergency Medicine Milestone Project, a framework for assessing competencies, has been used as a method of providing focused resident feedback. However, the emergency medicine milestones do not include specific objective data about resident clinical efficiency and productivity, and studies have shown that milestone-based feedback does not improve resident satisfaction with the feedback process. We examined whether providing performance metric reports to resident physicians improves their satisfaction with the feedback process and their clinical performance. Methods:We conducted a three-phase stepped-wedge randomized pilot study of emergency medicine residents at a single, urban academic site. In phase 1, all residents received traditional feedback; in phase 2, residents were randomized to receive traditional feedback (control group) or traditional feedback with performance metric reports (intervention group); and in phase 3, all residents received monthly performance metric reports and traditional feedback. To assess resident satisfaction with the feedback process, surveys using 6-point Likert scales were administered at each study phase and analyzed using two-sample t-tests. Analysis of variance in repeated measures was performed to compare impact of feedback on resident clinical performance, specifically patient treatment time (PTT) and patient visits per hour. Results:Forty-one residents participated in the trial of which 21 were randomized to the intervention group and 20 in the control group. Ninety percent of residents liked receiving the report and 74% believed that it better prepared them for expectations of becoming an attending physician. Additionally, residents randomized to the intervention group reported higher satisfaction (p = 0.01) with the quality of the feedback compared to residents in the control group. However, receiving performance metric reports, regardless of study phase or postgraduate year status, did not affect clinical performance, specifically PTT (183 minutes vs. 177 minutes, p = 0.34) or patients visits per hour (0.99 vs. 1.04, p = 0.46). Conclusions:While feedback with performance metric reports did not improve resident clinical performance, resident physicians were more satisfied with the feedback process, and a majority of residents expressed liking the reports and felt that it better prepared them to become attending physicians. Residency training programs could consider augmenting feedback with performance metric reports to aide in the transition from resident to attending physician.

SUBMITTER: Mamtani M

PROVIDER: S-EPMC6795364 | biostudies-literature | 2019 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Feedback With Performance Metric Scorecards Improves Resident Satisfaction but Does Not Impact Clinical Performance.

Mamtani Mira M Shofer Frances S FS Sackeim Alexander A Conlon Lauren L Scott Kevin K Mills Angela M AM

AEM education and training 20190520 4

<h4>Objectives</h4>The Emergency Medicine Milestone Project, a framework for assessing competencies, has been used as a method of providing focused resident feedback. However, the emergency medicine milestones do not include specific objective data about resident clinical efficiency and productivity, and studies have shown that milestone-based feedback does not improve resident satisfaction with the feedback process. We examined whether providing performance metric reports to resident physicians ...[more]

PMID: 31637349

Similar Datasets

Project description:Study objectiveWith the focus of patient-centered care in healthcare organizations, patient satisfaction plays an increasingly important role in healthcare quality measurement. We sought to determine whether an automated patient satisfaction survey could be effectively used to identify outlying anesthesiologists.DesignRetrospective Observational Study.SettingVanderbilt University Medical Center (VUMC).MeasurementsPatient satisfaction data were obtained between October 24, 2016 and November 1, 2017. A multivariable ordered probit regression was conducted to evaluate the relationship between the mean scores of responses to Likert-scale questions on SurveyVitals' Anesthesia Patient Satisfaction Questionnaire 2. Fixed effects included demographics, clinical variables, providers and surgeons. Hypothesis tests to compare each individual anesthesiologist with the median-performing anesthesiologist were conducted.Main resultsWe analyzed 10,528 surveys, with a 49.5% overall response rate. Younger patient (odds ratio (OR) 1.011 [per year of age]; 95% confidence interval (CI) 1.008 to 1.014; p < 0.001), regional anesthesia (versus general anesthesia) (OR 1.695; 95% CI 1.186 to 2.422; p = 0.004) and daytime surgery (versus nighttime surgery) (OR 1.795; 95% CI 1.091 to 2.959; p = 0.035) were associated with higher satisfaction scores. Compared with the median-ranked anesthesiologist, we found the adjusted odds ratio for an increase in satisfaction score ranged from 0.346 (95% CI 0.158 to 0.762) to 1.649 (95% CI 0.687 to 3.956) for the lowest and highest scoring providers, respectively. Only 10.10% of anesthesiologists at our institution had an odds ratio for satisfaction with a 95% CI not inclusive of 1.ConclusionsPatient satisfaction is impacted by multiple factors. There was very little information in patient satisfaction scores to discriminate the providers, after adjusting for confounding. While patient satisfaction scores may facilitate identification of extreme outliers among anesthesiologists, there is no evidence that this metric is useful for the routine evaluation of individual provider performance.

Project description:IntroductionThe discharge conversation is a critical component of the emergency department encounter. Studies suggest that emergency medicine (EM) residency education is deficient in formally training residents on the patient discharge conversation. Our goal was to assess the proficiency of EM residents in addressing essential elements of a comprehensive discharge conversation; identify which components of the discharge conversation are omitted; introduce "DC HOME," a standardized discharge mnemonic; and determine whether its implementation improved resident performance and patient satisfaction.MethodsThis was a prospective observational pre- and post-intervention study done by convenience sampling of 400 resident discharge encounters. Resident physicians were observed by attending physicians who completed an evaluation, answering "yes" or "no" as to whether residents addressed six components of a comprehensive discharge. The six components include the following: diagnosis; care rendered; health and lifestyle modifications; obstacles after discharge; medications; and expectations - or "DC HOME." Didactics introducing the mnemonic "DC HOME" was provided to resident physicians. Patient feedback and satisfaction were collected after each encounter, and we recorded differences between pre-intervention and post-intervention encounters.ResultsResident physicians improved significantly in all six components of "DC HOME" from pre-and-post intervention: discharge diagnosis (P = 0.0036) and the remaining five components (P<0.0001). There was a statistically significant improvement in patients' perception for health and lifestyle modifications, obstacles after discharge, medications, expectations after discharge (P<0.0001), and discharge diagnosis (P = 0.0029). Patient satisfaction scores improved significantly (P = 0.005). Time spent with patients during discharge increased from 2 minutes and 42 seconds to 4 minutes and 4 seconds (P<0.0001).ConclusionEM residents frequently omit key components of the discharge conversation. The implementation of the "DC HOME" discharge mnemonic improves resident discharge performance, patient perception, and overall patient satisfaction.

Project description:PurposeResidency training environments can differ significantly; therefore, resident satisfaction may vary widely among programs. Here, we sought to examine several variables in program satisfaction through a survey of radiation oncology (RO) trainees in the United States.Methods and materialsAn anonymous, institutional review board-approved, internet-based survey was developed and distributed to U.S. residents in RO in September 2016. This email-based survey assessed program-specific factors with regard to workload, work-life balance, and education as well as resident-specific factors such as marital status and postgraduate year. Binomial multivariable regression assessed the correlations between these factors and the endpoint of resident-reported likelihood of selecting an alternative RO residency program if given the choice again.ResultsA total of 215 residents completed the required survey sections, representing 29.3% of U.S. RO residents. When asked whether residency allowed for an adequate balance between work and personal life, the majority of residents (75.6%) agreed or strongly agreed, but a minority (9.3%) did not feel that residency allowed for sufficient time for personal life. The majority of residents (69.7%) indicated that they would choose the same residency program again, but 12.2% would have made a different choice. Almost three-fourths of residents (73.0%) felt that faculty and staff cared about the educational success of residents, but 9.27% did not. Binomial multivariable regression revealed that senior residents (odds ratio: 6.70; 95% confidence interval, 2.20-22.4) were more likely to desire a different residency program. In contrast, residents who reported constructive feedback use by the residency program (odds ratio:0.22; 95% confidence interval, 0.06-0.91) were more satisfied with their program choice.ConclusionsMost RO residents reported satisfaction with their choice of residency program, but seniors had higher rates of dissatisfaction. Possible interventions to improve professional satisfaction include incorporating constructive resident feedback to enhance the program. The potential impact of job market pressures on seniors should be further explored.

Project description:IntroductionThe Accreditation Council for Graduate Medical Education (ACGME) specifically notes multisource feedback (MSF) as a recommended means of resident assessment in the emergency medicine (EM) Milestones. High-fidelity simulation is an environment wherein residents can receive MSF from various types of healthcare professionals. Previously, the Queen's Simulation Assessment Tool (QSAT) has been validated for faculty to assess residents in five categories: assessment; diagnostic actions; therapeutic actions; interpersonal communication, and overall assessment. We sought to determine whether the QSAT could be used to provide MSF using a standardized simulation case.MethodsProspectively after institutional review board approval, residents from a dual ACGME/osteopathic-approved postgraduate years (PGY) 1-4 EM residency were consented for participation. We developed a standardized resuscitation after overdose case with specific 1-5 Likert anchors used by the QSAT. A PGY 2-4 resident participated in the role of team leader, who completed a QSAT as self-assessment. The team consisted of a PGY-1 peer, an emergency medical services (EMS) provider, and a nurse. Two core faculty were present to administer the simulation case and assess. Demographics were gathered from all participants completing QSATs. We analyzed QSATs by each category and on cumulative score. Hypothesis testing was performed using intraclass correlation coefficients (ICC), with 95% confidence intervals. Interpretation of ICC results was based on previously published definitions.ResultsWe enrolled 34 team leader residents along with 34 nurses. A single PGY-1, a single EMS provider and two faculty were also enrolled. Faculty provided higher cumulative QSAT scores than the other sources of MSF. QSAT scores did not increase with team leader PGY level. ICC for inter-rater reliability for all sources of MSF was 0.754 (0.572-0.867). Removing the self-evaluation scores increased inter-rater reliability to 0.838 (0.733-0.910). There was lesser agreement between faculty and nurse evaluations than from the EMS or peer evaluation.ConclusionIn this single-site cohort using an internally developed simulation case, the QSAT provided MSF with excellent reliability. Self-assessment decreases the reliability of the MSF, and our data suggest self-assessment should not be a component of MSF. Use of the QSAT for MSF may be considered as a source of data for clinical competency committees.

Dataset Information

Feedback With Performance Metric Scorecards Improves Resident Satisfaction but Does Not Impact Clinical Performance.

Publications

Feedback With Performance Metric Scorecards Improves Resident Satisfaction but Does Not Impact Clinical Performance.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets