Dataset Information

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure.

ABSTRACT: BACKGROUND:Outcomes of surgical trials hinge on surgeon selection and their underlying expertise. Assessment of expertise is paramount. We investigated whether surgeons' performance measured by the fundamentals of laparoscopic surgery (FLS) assessment program could predict their performance in a surgical trial. METHODS:As part of a prospective multi-institutional study of minimally invasive inguinal lymphadenectomy (MILND) for melanoma, surgical oncologists with no prior MILND experience underwent pre-trial FLS assessment. Surgeons completed MILND training, began enrolling patients, and submitted videos of each MILND case performed. Videos were scored with the global operative assessment of laparoscopic skills (GOALS) tool. Associations between baseline FLS scores and participant's trial performance metrics were assessed. RESULTS:Twelve surgeons enrolled patients; their median total baseline FLS score was 332 (range 275-380, max possible 500, passing >270). Participants enrolled 87 patients in the study (median 6 per surgeon, range 1-24), of which 72 (83%) videos were adequate for scoring. Baseline GOALS score was 17.1 (range 9.6-21.2, max possible score 30). Inter-rater reliability was excellent (ICC = 0.85). FLS scores correlated with improved GOALS scores (r = 0.57, p = 0.05) and with decreased operative time (r = -0.6, p = 0.02). No associations were found with the degree of patient recruitment (r = 0.02, p = 0.7), lymph node count (r = 0.01, p = 0.07), conversion rate (r = -0.06, p = 0.38) or major complications(r = -0.14, p = 0.6). CONCLUSIONS:FLS skill assessment of surgeons prior to their enrollment in a surgical trial is feasible. Although better FLS scores predicted improved operative performance and operative time, other trial outcome measures showed no difference. Our findings have implications for the documentation of laparoscopic expertise of surgeons in practice and may allow more appropriate selection of surgeons to participate in clinical trials.

SUBMITTER: Zendejas B

PROVIDER: S-EPMC6201269 | biostudies-literature | 2017 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure.

Zendejas Benjamin B Jakub James W JW Terando Alicia M AM Sarnaik Amod A Ariyan Charlotte E CE Faries Mark B MB Zani Sabino S Neuman Heather B HB Wasif Nabil N Farma Jeffrey M JM Averbook Bruce J BJ Bilimoria Karl Y KY Tyler Douglas D Brady Mary Sue MS Farley David R DR

Surgical endoscopy 20161207 8

<h4>Background</h4>Outcomes of surgical trials hinge on surgeon selection and their underlying expertise. Assessment of expertise is paramount. We investigated whether surgeons' performance measured by the fundamentals of laparoscopic surgery (FLS) assessment program could predict their performance in a surgical trial.<h4>Methods</h4>As part of a prospective multi-institutional study of minimally invasive inguinal lymphadenectomy (MILND) for melanoma, surgical oncologists with no prior MILND exp ...[more]

PMID: 27928664

Similar Datasets

Project description:The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Project description:ImportanceSurgical coaching is maturing as a tangible strategy for surgeons' continuing professional development. Resources to spread this innovation are not yet widely available.ObjectiveTo identify surgeon-derived implementation recommendations for surgical coaching programs from participants' exit interviews and ratings of their coaching interactions.Design, setting, and participantsThis qualitative analysis of the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, a quality improvement intervention, was conducted at 4 US academic medical centers. Participants included 46 practicing surgeons. The SCOPE program ran from December 7, 2018, to October 31, 2019. Data were analyzed from November 1, 2019, to January 31, 2020.InterventionsSurgeons were assigned as either a coach or a coachee, and each coach was paired with 1 coachee by a local champion who knew the surgeons professionally. Coaching pairs underwent training and were instructed to complete 3 coaching sessions-consisting of preoperative goal setting, intraoperative observation, and postoperative debriefing-focused on intraoperative performance.Main outcomes and measuresThemes from the participants' exit interviews covering 3 major domains: (1) describing the experience, (2) coach-coachee relationship, and (3) facilitators and barriers to implementing surgical coaching. Surgeons' responses were stratified by the net promoter score (NPS), a scale ranging from 0 to 10 points, indicating how likely they were to recommend their coaching session to others, with 9 to 10 indicating promoters; 7 to 8, passives; and 0 to 6, detractors.ResultsAmong the 46 participants (36 men [78.3%]), 23 were interviewed (50.0%); thematic saturation was reached with 5 coach-coachee pairs (10 interviews). Overall, coaches and coachees agreed on key implementation recommendations for surgical coaching, including how to optimize coach-coachee relationships and facilitate productive coaching sessions. The NPS categories were associated with how participants experienced their own coaching sessions. Specifically, participants who reported excellent first sessions, had a coaching partner in the same clinical specialty, and were transparent about each other's intentions in the program tended to be promoters. Participants who described suboptimal first sessions, less clinical overlap, and unclear goals with their partner were more likely detractors.Conclusions and relevanceThese exit interviews with practicing surgeons offer critical insights for addressing cultural barriers and practical challenges for successful implementation of peer coaching programs focused on surgical performance improvement. With empirical evidence on optimizing coach-coachee relationships and facilitating participants' experience, organizations can establish effective coaching programs that enable meaningful continuous professional development for surgeons and ultimately enhance patient care.

Project description:PurposeSurgeons' skill in the operating room is a major determinant of patient outcomes. Assessment of surgeons' skill is necessary to improve patient outcomes and quality of care through surgical training and coaching. Methods for video-based assessment of surgical skill can provide objective and efficient tools for surgeons. Our work introduces a new method based on attention mechanisms and provides a comprehensive comparative analysis of state-of-the-art methods for video-based assessment of surgical skill in the operating room.MethodsUsing a dataset of 99 videos of capsulorhexis, a critical step in cataract surgery, we evaluated image feature-based methods and two deep learning methods to assess skill using RGB videos. In the first method, we predict instrument tips as keypoints and predict surgical skill using temporal convolutional neural networks. In the second method, we propose a frame-wise encoder (2D convolutional neural network) followed by a temporal model (recurrent neural network), both of which are augmented by visual attention mechanisms. We computed the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and predictive values through fivefold cross-validation.ResultsTo classify a binary skill label (expert vs. novice), the range of AUC estimates was 0.49 (95% confidence interval; CI = 0.37 to 0.60) to 0.76 (95% CI = 0.66 to 0.85) for image feature-based methods. The sensitivity and specificity were consistently high for none of the methods. For the deep learning methods, the AUC was 0.79 (95% CI = 0.70 to 0.88) using keypoints alone, 0.78 (95% CI = 0.69 to 0.88) and 0.75 (95% CI = 0.65 to 0.85) with and without attention mechanisms, respectively.ConclusionDeep learning methods are necessary for video-based assessment of surgical skill in the operating room. Attention mechanisms improved discrimination ability of the network. Our findings should be evaluated for external validity in other datasets.

Project description:BackgroundAdvanced surgical techniques, such as total laparoscopic hysterectomy, are often challenging to acquire beyond fellowship training programs for practicing obstetrician-gynecologists. A lack of formative data currently exist for continuing medical education programs, limiting our understanding of how improvement in surgical skills and training programs occur.ObjectiveThis study aimed to investigate how practicing obstetrician-gynecologists acting as trainees experience a program that aims to teach them total laparoscopic hysterectomy, and to assess whether their surgical skills improve according to data from formative assessment tools and qualitative data from open-ended survey questions and in-depth interviews.Study designWe report a process analysis of formative data collected during a pilot implementation trial of a surgical training program targeting practicing obstetrician-gynecologists. Eleven consultant obstetrician-gynecologists and 4 experienced surgical mentors participated in 4 hospitals in Queensland, Australia. Total laparoscopic hysterectomy was performed in 700 patients over the course of the study. A total laparoscopic hysterectomy surgical mentorship training program of 10 training days with up to 3 total laparoscopic hysterectomy procedures per day was performed. Both the obstetrician-gynecologists and the surgical mentor completed a formative assessment questionnaire analyzing the trainee's performance after each surgical procedure. Mentors were formatively assessed by the Structured Training Trainer Assessment Report (STTAR) and at the completion of the study by the mini-STTAR, a summative assessment of quality of mentorship. Obstetrician-gynecologists, mentors, hospital leaders, and surgical administrative staff participated in qualitative interviews about the training program.ResultsOver time, there was a demonstrated improvement in trainee performance reported by both mentors and trainees in all competency assessment tool domains as the case number increased, with mentors consistently rating trainees' performance higher than the trainees themselves. Most trainees were satisfied with their mentor in all 31 areas during formative assessment, and at the end of the training, structure, attributes, and role modeling were all rated high (average score >4.5; range, 3.79-5.00), whereas training behavior was rated slightly lower at 4.1 (range, 3.79-4.45). Qualitative interviews demonstrated that the trainees found the training to be a beneficial, hands-on experience.ConclusionFormative assessment clearly documented improvement in surgical skills during a total laparoscopic hysterectomy training program for consultant obstetrician-gynecologists.

Dataset Information

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure.

Publications

Laparoscopic skill assessment of practicing surgeons prior to enrollment in a surgical trial of a new laparoscopic procedure.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets