Dataset Information

Association of Surgical Skill Assessment With Clinical Outcomes in Cancer Surgery.

ABSTRACT:

Importance

Complex surgical interventions are inherently prone to variation yet they are not objectively measured. The reasons for outcome differences following cancer surgery are unclear.

Objective

To quantify surgical skill within advanced laparoscopic procedures and its association with histopathological and clinical outcomes.

Design, setting, and participants

This analysis of data and video from the Australasian Laparoscopic Cancer of Rectum (ALaCaRT) and 2-dimensional/3-dimensional (2D3D) multicenter randomized laparoscopic total mesorectal excision trials, which were conducted at 28 centers in Australia, the United Kingdom, and New Zealand, was performed from 2018 to 2019 and included 176 patients with clinical T1 to T3 rectal adenocarcinoma 15 cm or less from the anal verge. Case videos underwent blinded objective analysis using a bespoke performance assessment tool developed with a 62-international expert Delphi exercise and workshop, interview, and pilot phases.

Interventions

Laparoscopic total mesorectal excision undertaken with curative intent by 34 credentialed surgeons.

Main outcomes and measures

Histopathological (plane of mesorectal dissection, ALaCaRT composite end point success [mesorectal fascial plane, circumferential margin, ≥1 mm; distal margin, ≥1 mm]) and 30-day morbidity. End points were analyzed using surgeon quartiles defined by tool scores.

Results

The laparoscopic total mesorectal excision performance tool was produced and shown to be reliable and valid for the specialist level (intraclass correlation coefficient, 0.889; 95% CI, 0.832-0.926; P < .001). A substantial variation in tool scores was recorded (range, 25-48). Scores were associated with the number of intraoperative errors, plane of mesorectal dissection, and short-term patient morbidity, including the number and severity of complications. Upper quartile-scoring surgeons obtained excellent results compared with the lower quartile (mesorectal fascial plane: 93% vs 59%; number needed to treat [NNT], 2.9, P = .002; ALaCaRT end point success, 83% vs 58%; NNT, 4; P = .03; 30-day morbidity, 23% vs 50%; NNT, 3.7; P = .03).

Conclusions and relevance

Intraoperative surgical skill can be objectively and reliably measured in complex cancer interventions. Substantial variation in technical performance among credentialed surgeons is seen and significantly associated with clinical and pathological outcomes.

SUBMITTER: Curtis NJ

PROVIDER: S-EPMC7203671 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo test whether crowdsourced lay raters can accurately assess cataract surgical skills.DesignTwo-armed study: independent cross-sectional and longitudinal cohorts.SettingWashington University Department of Ophthalmology.Participants and methodsSixteen cataract surgeons with varying experience levels submitted cataract surgery videos to be graded by 5 experts and 300+ crowdworkers masked to surgeon experience. Cross-sectional study: 50 videos from surgeons ranging from first-year resident to attending physician, pooled by years of training. Longitudinal study: 28 videos obtained at regular intervals as residents progressed through 180 cases. Surgical skill was graded using the modified Objective Structured Assessment of Technical Skill (mOSATS). Main outcome measures were overall technical performance, reliability indices, and correlation between expert and crowd mean scores.ResultsExperts demonstrated high interrater reliability and accurately predicted training level, establishing construct validity for the modified OSATS. Crowd scores were correlated with (r = 0.865, p < 0.0001) but consistently higher than expert scores for first, second, and third-year residents (p < 0.0001, paired t-test). Longer surgery duration negatively correlated with training level (r = -0.855, p < 0.0001) and expert score (r = -0.927, p < 0.0001). The longitudinal dataset reproduced cross-sectional study findings for crowd and expert comparisons. A regression equation transforming crowd score plus video length into expert score was derived from the cross-sectional dataset (r2 = 0.92) and demonstrated excellent predictive modeling when applied to the independent longitudinal dataset (r2 = 0.80). A group of student raters who had edited the cataract videos also graded them, producing scores that more closely approximated experts than the crowd.ConclusionsCrowdsourced rankings correlated with expert scores, but were not equivalent; crowd scores overestimated technical competency, especially for novice surgeons. A novel approach of adjusting crowd scores with surgery duration generated a more accurate predictive model for surgical skill. More studies are needed before crowdsourcing can be reliably used for assessing surgical proficiency.

Project description:The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Project description:ObjectiveTo examine trends and associated characteristics and outcomes of minimally invasive surgery (MIS) for women with early-stage ovarian cancer.MethodsThe National Inpatient Sample was queried to examine early-stage ovarian cancer treated with MIS from 2001 to 2011. Annualized hospital surgical volume was defined in the unweighted model as the average number of procedures performed per year in which at least one case was performed. Trends, characteristics, and outcomes related to MIS use were assessed in the weighted model.ResultsAmong 73,707 oophorectomy cases, there were 4822 (6.5%) MIS cases. Utilization of MIS increased from 3.9% to 13.5% from 2001 to 2011 (3.5-fold increase, P < 0.001), and the number of MIS-offering centers also increased from 10.6% to 36.2% (3.4-fold increase, P < 0.001). MIS was associated with a decreased complication rate (20.3% versus 35.4%) and shorter hospital stay (median, 2 versus 4 days) compared to laparotomy (both, P < 0.001). Of the 472 hospitals at which MIS was performed, the majority were minimum-volume with one MIS oophorectomy per year (340 [72.0%], n = 1929 [40.0%]), followed by mid-volume (85 [18.0%], n = 1272 [26.4%]) and topdecile-volume (47 [10.0%] hospitals, n = 1621 [33.6%]). The topdecile-volume group had the highest rate of lymphadenectomy compared to other groups (62.2% versus 39.2-55.1%, P < 0.05). On multivariable analysis, a one increment increase in annualized hospital surgical volume was associated with an 11% decrease in multiple complications (adjusted-odds ratio 0.89, 95% confidence interval 0.82-0.97, P = 0.006).ConclusionUtilization of MIS for early-stage ovarian cancer has significantly increased in the United States in 2000s. In 2011, one in eight surgeries performed for early ovarian cancer were performed via MIS. MIS procedures performed at hospitals with a higher surgical volume may be associated with improved short-term perioperative outcomes.

Project description:ImportanceAnatomical scoring systems for coronary artery disease, such as the SYNTAX (Synergy Between Percutaneous Coronary Intervention [PCI] With Taxus and Cardiac Surgery) score, are well established tools for understanding patient risk. However, they are cumbersome to compute manually for large data sets, limiting their use across broad and varied cohorts.ObjectiveTo adapt an anatomical scoring system for use with registry data, allowing facile and automatic calculation of scores and association with clinical outcomes among patients undergoing percutaneous or surgical revascularization.Design, setting, and participantsThis cross-sectional observational cohort study involved procedures performed in all cardiac catheterization laboratories in the largest integrated health care system in the United States, the Veterans Affairs (VA) Healthcare System. Patients undergoing coronary angiography in the VA Healthcare System followed by percutaneous or surgical revascularization within 90 days were observed and data were analyzed from January 1, 2010, through September 30, 2017.Main outcomes and measuresAn anatomical scoring system for coronary artery disease complexity before revascularization was simplified and adapted to data from the VA Clinical Assessment, Reporting, and Tracking Program. The adjusted association between quantified anatomical complexity and major adverse cardiovascular and cerebrovascular events (MACCEs), including death, myocardial infarction, stroke, and repeat revascularization, was assessed for patients undergoing percutaneous or surgical revascularization.ResultsA total of 50 226 patients (49 359 men [98.3%]; mean [SD] age, 66 [9] years) underwent revascularization during the study period, with 34 322 undergoing PCI and 15 904 undergoing coronary artery bypass grafting (CABG). After adjustment, the highest tertile of anatomical complexity was associated with increased hazard of MACCEs (adjusted hazard ratio [HR], 2.12; 95% CI, 2.01-2.23). In contrast, the highest tertile of anatomical complexity among patients undergoing CABG was not independently associated with overall MACCEs (adjusted HR, 1.04; 95% CI, 0.92-1.17), and only repeat revascularization was associated with increasing complexity (adjusted HR, 1.34; 95% CI, 1.06-1.70) in this subgroup.Conclusions and relevanceThese findings suggest that an automatically computed score assessing anatomical complexity can be used to assess longitudinal risk for patients undergoing revascularization. This simplified scoring system appears to be an alternative tool for understanding longitudinal risk across large data sets.