Dataset Information

How does multisource feedback influence residency training? A qualitative case study.

ABSTRACT:

Introduction

Multisource feedback (MSF), also called 360-degree assessment, is one form of assessment used in postgraduate training. However, there is an ongoing discussion on its value, because the factors that influence the impact of MSF and the main impact of MSF are not fully understood. In this study, we investigated both the influencing factors and the impact of MSF on residency training.

Methods

We conducted a qualitative case study within the boundaries of the residency training for paediatricians and paediatric surgeons at a University Hospital. We collected data from seven focus group interviews with stakeholders of MSF (residents, raters and supervisors). By performing a reflexive thematic analysis, we extracted the influencing factors and the impact of MSF.

Results

We found seven influencing factors: MSF is facilitated by the announcement of a clear goal of MSF, the training of raters on the MSF instrument, a longitudinal approach of observation, timing not too early and not too late during the rotation, narrative comments as part of the ratings, the residents' self-assessment and a supervisor from the same department. We found three themes on the impact of MSF: MSF supports the professional development of residents, enhances interprofessional teamwork and increases the raters' commitment to the training of residents.

Conclusion

This study illuminates the influencing factors and impact of MSF on residency training. We offer novel recommendations on the continuity of observation, the timing during rotations and the role of the supervisor. Moreover, by discussing our results through the lens of identity formation theory, this work advances our conceptual understanding of MSF. We propose identity formation theory as a framework for future research on MSF to leverage the potential of MSF in residency training.

SUBMITTER: Hennel EK

PROVIDER: S-EPMC9314722 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectivesTo examine a narrative multisource feedback (MSF) instrument concerning feasibility, quality of narrative comments, perceptions of users (face validity), consequential validity, discriminating capacity and number of assessors needed.DesignQualitative text analysis supplemented by quantitative descriptive analysis.SettingInternal Medicine Departments in Zealand, Denmark.Participants48 postgraduate trainees in internal medicine specialties, 1 clinical supervisor for each trainee and 376 feedback givers (respondents).InterventionThis study examines the use of an electronic, purely narrative MSF instrument. After the MSF process, the trainee and the supervisor answered a postquestionnaire concerning their perception of the process. The authors coded the comments in the MSF reports for valence (positive or negative), specificity, relation to behaviour and whether the comment suggested a strategy for improvement. Four of the authors independently classified the MSF reports as either 'no reasons for concern' or 'possibly some concern', thereby examining discriminating capacity. Through iterative readings, the authors furthermore tried to identify how many respondents were needed in order to get a reliable impression of a trainee.ResultsOut of all comments coded for valence (n=1935), 89% were positive and 11% negative. Out of all coded comments (n=4684), 3.8% were suggesting ways to improve. 92% of trainees and supervisors preferred a narrative MSF to a numerical MSF, and 82% of the trainees discovered performance in need of development, but only 53% had made a specific plan for development. Kappa coefficients for inter-rater correlations between four authors were 0.7-1. There was a significant association (p<0.001) between the number of negative comments and the qualitative judgement by the four authors. It was not possible to define a specific number of respondents needed.ConclusionsA purely narrative MSF contributes with educational value and experienced supervisors can discriminate between trainees' performances based on the MSF reports.

Project description:ObjectivesTrauma resuscitations require competence in both clinical and nonclinical skills but these can be difficult to observe and assess. Multisource feedback (MSF) is workplace-based, involves the direct observation of learners, and can provide feedback on nonclinical skills. We sought to compare and contrast the priorities of multidisciplinary trauma team members when assessing resident trauma team captain (TTC) performance. Additionally, we aimed to describe the nature of the assessment and perceived the utility of incorporating MSF into the trauma context.MethodsA convenience sample of 10 trauma team activations were observed. Following each activation, the attending physician trauma team leader (TTL), TTC, and a registered nurse (RN) participated in a semistructured interview. MSF was not provided to the TTC for the purpose of this study because MSF was not part of the assessment process of TTCs at the time of this study and maintaining anonymity may have encouraged more honest interview responses. Transcripts from each assessor group (TTL, TTC, RN) were coded and assigned to one of the five crisis resource management skills: leadership, communication, situational awareness, resource utilization, and problem-solving. Comments were also coded as positive, negative, or neutral as interpreted by the coder.ResultsAll assessor groups mentioned communication skills most frequently. After communication, the RN and TTC groups commented on situational awareness most frequently, comprising 15 and 29% of their total responses, respectively, whereas 31% of the TTL comments focused on leadership skills. The RN and TTL groups provided positive assessments, with 51 and 42% of their respective comments coded as positive. Forty-five percent of self-assessment comments in the TTC group were negative. All (100%) of the TTC and TTL respondents felt that incorporating MSF would add to the quality of feedback, only 66% of the RN group felt that way.ConclusionsWe found that each assessor group brings a unique focus and perspective to the assessment of resident TTC performance. The future inclusion of MSF in the trauma team context has the potential to enhance the learning environment in a clinical arena that is difficult to directly observe and assess.

Project description:BACKGROUND:Handoffs are a complex procedure whose success relies on mutual discussion rather than simple information transfer. Particularly among trainees, handoffs present major opportunities for medical error. Previous research has explored best practices and pitfalls in general handoff education but has not discussed barriers specific to anesthesiology residents. This study characterizes the experiences of residents in anesthesiology as they learn handoff technique in order to inform strategies for teaching this important component of perioperative care. METHODS:In 2016, we conducted a semi-structured interview study of 30 anesthesia residents across all three postgraduate years at a major academic hospital. Interviews were coded by two coders using a grounded theory approach and an iterative process designed to enhance reliability and validity. RESULTS:Residents cited lack of consistency as a major impediment to proper handoff education. They found the impact of lectures and written materials to be limited. The level of guidance and direction they received from one-to-one attendings was described as highly variable. Residents' comfort in executing handoffs was heavily dependent on location and situation. They felt that coordination among the parties involved in the handoff was difficult to achieve, causing confusion about the importance of handoffs as well as proper protocol. Finally, residents offered opinions on when handoff education should occur during the residency and had several recommendations for its improving, including standardization of key handoff topics. CONCLUSIONS:In a single center study of anesthesiology resident handoff education, residents exhibited confusion related to a perceived disconnect between the stated importance of effective handoffs and a lack of consensus on proper handoff technique. Standardization of curriculum and framing expectations has the potential to enhance resident handoff training in academic anesthesia departments.