Dataset Information

Development and Assessment of the Wisconsin Surgical Coaching Rubric.

ABSTRACT:

Importance

Surgical coaching continues to gain momentum as an innovative method for continuous professional development. A tool to measure the performance of a surgical coach is needed to provide formative feedback to coaches for continued skill development and to assess the fidelity of a coaching intervention for future research and dissemination.

Objective

To evaluate the validity of the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool to assess the performance of a peer surgical coach.

Design, setting, and participants

Surgical coaching sessions from November 2014 through February 2018 conducted by 2 statewide peer surgical coaching programs were audio recorded and transcribed. Twelve raters used WiSCoR to rate the performance of the surgical coach for each session. The study included peer surgical coaches in the Wisconsin Surgical Coaching Program (n = 8) and the Michigan Bariatric Surgery Collaborative coaching program (n = 15). The data were analyzed in 2019.

Interventions or exposures

Use of WiSCoR to rate peer surgical coaching sessions.

Main outcomes and measures

There were 282 WiSCoR ratings from the 106 coaching sessions included in the study. WiSCoR was evaluated using a framework, including inter-rater reliability assessed with Gwet weighted agreement coefficent. Descriptive statistics of WiSCoR were calculated.

Results

Eight coaches (35%) and 11 coachees (29%) were from the Wisconsin Surgical Program and 15 coaches (65%) and 27 coachees (71%) were from the Michigan Bariatric Surgery Collaborative. The validity of WiSCoR is supported by high interrater reliability (Gwet weighted agreement coefficient, 0.87) as well as a weakly positive correlation of WiSCoR to coachee ratings of coaches (r = 0.22; P = .04), rigorous content development, consistent rater training, and the association of WiSCoR with coach and coaching program development. The mean (SD) overall coach performance rating using WiSCoR was 3.23 (0.82; range, 1-5).

Conclusions and relevance

WiSCoR is a reliable measure that can assess the performance of a surgical coach, inform fidelity to coaching principles, and provide formative feedback to surgical coaches. While coachee ratings may reflect coachee satisfaction, they are not able to determine the quality of a coach.

SUBMITTER: Vande Walle KA

PROVIDER: S-EPMC7177649 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background:Documentation of information in the health record by pharmacists is vital to patient care. Failure to document, or failure to document appropriately, may have negative effects on patients. Objectives:The primary objective was to determine pharmacists' competency in 18 elements of chart note documentation at 2 tertiary care centres and 1 rehabilitation centre. The secondary objectives were to quantify the number of episodes of documentation by pharmacists, to characterize the clinical activities associated with this documentation, and to determine whether there were differences in level of competency according to years of hospital experience, additional clinical training, and note type. Methods:This study used prospective audit methodology. Notes documented during a 6-week study period (February to April 2015) were assessed using a rubric specifically created to evaluate 18 of the essential elements of documentation, as defined by the Canadian Society of Hospital Pharmacists. Four of the elements of chart note documentation were assessed on a yes/no basis, and Bloom's taxonomy was used to define the level of competency for the other 14 elements. Values were assigned to each level and were then used to calculate mean scores. Results:A total of 115 pharmacist notes, created by 29 of the 35 pharmacists included in the study, were assessed. The mean competency score per pharmacist was 2.2 (standard deviation [SD] 0.3; maximum score 3). Elements of documentation with the highest level of competency were diplomatic tone, conciseness, and clarity; elements with the lowest level of competency were medication list, note title, monitoring, and drug-related problem statements. The most frequent note types concerned drug-related problems (50 [43%]), pharmacokinetics (25 [22%]), and patient education (19 [17%]). Levels of competency were highest for notes related to patient education (mean 2.4, SD 0.2) and lowest for notes concerning drug-related problems (mean 2.1, SD 0.3) and notes providing clarification (mean 2.1, SD 0.3). The level of competency was not significantly affected by additional clinical training or years of hospital experience. Conclusions:Pharmacists in this study documented concisely, clearly, and in a diplomatic tone; however, there was room for improvement in the frequency and elements of chart note documentation in the patient health record.

Project description:BackgroundLearning strategies are thinking tools that students can use to actively acquire information. Examples of learning strategies include mnemonics, charts, and maps. One strategy that may help students master the tsunami of information presented in medical school is the mind map learning strategy. Currently, there is no valid and reliable rubric to grade mind maps and this may contribute to their underutilization in medicine. Because concept maps and mind maps engage learners similarly at a metacognitive level, a valid and reliable concept map assessment scoring system was adapted to form the mind map assessment rubric (MMAR). The MMAR can assess mind map depth based upon concept-links, cross-links, hierarchies, examples, pictures, and colors. The purpose of this study was to examine interrater reliability of the MMAR.MethodsThis exploratory study was conducted at a US medical school as part of a larger investigation on learning strategies. Sixty-six (N = 66) first-year medical students were given a 394-word text passage followed by a 30-minute presentation on mind mapping. After the presentation, subjects were again given the text passage and instructed to create mind maps based upon the passage. The mind maps were collected and independently scored using the MMAR by 3 examiners. Interrater reliability was measured using the intraclass correlation coefficient (ICC) statistic. Statistics were calculated using SPSS version 12.0 (Chicago, IL).ResultsAnalysis of the mind maps revealed the following: concept-links ICC = .05 (95% CI, -.42 to .38), cross-links ICC = .58 (95% CI, .37 to .73), hierarchies ICC = .23 (95% CI, -.15 to .50), examples ICC = .53 (95% CI, .29 to .69), pictures ICC = .86 (95% CI, .79 to .91), colors ICC = .73 (95% CI, .59 to .82), and total score ICC = .86 (95% CI, .79 to .91).ConclusionThe high ICC value for total mind map score indicates strong MMAR interrater reliability. Pictures and colors demonstrated moderate to strong interrater reliability. We conclude that the MMAR may be a valid and reliable tool to assess mind maps in medicine. However, further research on the validity and reliability of the MMAR is necessary.

Project description:ImportanceSurgical coaching is maturing as a tangible strategy for surgeons' continuing professional development. Resources to spread this innovation are not yet widely available.ObjectiveTo identify surgeon-derived implementation recommendations for surgical coaching programs from participants' exit interviews and ratings of their coaching interactions.Design, setting, and participantsThis qualitative analysis of the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, a quality improvement intervention, was conducted at 4 US academic medical centers. Participants included 46 practicing surgeons. The SCOPE program ran from December 7, 2018, to October 31, 2019. Data were analyzed from November 1, 2019, to January 31, 2020.InterventionsSurgeons were assigned as either a coach or a coachee, and each coach was paired with 1 coachee by a local champion who knew the surgeons professionally. Coaching pairs underwent training and were instructed to complete 3 coaching sessions-consisting of preoperative goal setting, intraoperative observation, and postoperative debriefing-focused on intraoperative performance.Main outcomes and measuresThemes from the participants' exit interviews covering 3 major domains: (1) describing the experience, (2) coach-coachee relationship, and (3) facilitators and barriers to implementing surgical coaching. Surgeons' responses were stratified by the net promoter score (NPS), a scale ranging from 0 to 10 points, indicating how likely they were to recommend their coaching session to others, with 9 to 10 indicating promoters; 7 to 8, passives; and 0 to 6, detractors.ResultsAmong the 46 participants (36 men [78.3%]), 23 were interviewed (50.0%); thematic saturation was reached with 5 coach-coachee pairs (10 interviews). Overall, coaches and coachees agreed on key implementation recommendations for surgical coaching, including how to optimize coach-coachee relationships and facilitate productive coaching sessions. The NPS categories were associated with how participants experienced their own coaching sessions. Specifically, participants who reported excellent first sessions, had a coaching partner in the same clinical specialty, and were transparent about each other's intentions in the program tended to be promoters. Participants who described suboptimal first sessions, less clinical overlap, and unclear goals with their partner were more likely detractors.Conclusions and relevanceThese exit interviews with practicing surgeons offer critical insights for addressing cultural barriers and practical challenges for successful implementation of peer coaching programs focused on surgical performance improvement. With empirical evidence on optimizing coach-coachee relationships and facilitating participants' experience, organizations can establish effective coaching programs that enable meaningful continuous professional development for surgeons and ultimately enhance patient care.