Dataset Information

Development and validation of the Surgical Outcome Risk Tool (SORT).

ABSTRACT:

Background

Existing risk stratification tools have limitations and clinical experience suggests they are not used routinely. The aim of this study was to develop and validate a preoperative risk stratification tool to predict 30-day mortality after non-cardiac surgery in adults by analysis of data from the observational National Confidential Enquiry into Patient Outcome and Death (NCEPOD) Knowing the Risk study.

Methods

The data set was split into derivation and validation cohorts. Logistic regression was used to construct a model in the derivation cohort to create the Surgical Outcome Risk Tool (SORT), which was tested in the validation cohort.

Results

Prospective data for 19?097 cases in 326 hospitals were obtained from the NCEPOD study. Following exclusion of 2309, details of 16?788 patients were analysed (derivation cohort 11?219, validation cohort 5569). A model of 45 risk factors was refined on repeated regression analyses to develop a model comprising six variables: American Society of Anesthesiologists Physical Status (ASA-PS) grade, urgency of surgery (expedited, urgent, immediate), high-risk surgical specialty (gastrointestinal, thoracic, vascular), surgical severity (from minor to complex major), cancer and age 65?years or over. In the validation cohort, the SORT was well calibrated and demonstrated better discrimination than the ASA-PS and Surgical Risk Scale; areas under the receiver operating characteristic (ROC) curve were 0·91 (95 per cent c.i. 0·88 to 0·94), 0·87 (0·84 to 0·91) and 0·88 (0·84 to 0·92) respectively (P?ConclusionThe SORT allows rapid and simple data entry of six preoperative variables, and provides a percentage mortality risk for individuals undergoing surgery.

SUBMITTER: Protopapa KL

PROVIDER: S-EPMC4240514 | biostudies-literature | 2014 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and validation of the Surgical Outcome Risk Tool (SORT).

Protopapa K L KL Simpson J C JC Smith N C E NC Moonesinghe S R SR

The British journal of surgery 20141201 13

<h4>Background</h4>Existing risk stratification tools have limitations and clinical experience suggests they are not used routinely. The aim of this study was to develop and validate a preoperative risk stratification tool to predict 30-day mortality after non-cardiac surgery in adults by analysis of data from the observational National Confidential Enquiry into Patient Outcome and Death (NCEPOD) Knowing the Risk study.<h4>Methods</h4>The data set was split into derivation and validation cohorts ...[more]

PMID: 25388883

Similar Datasets

Project description:ImportanceGrowing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations.ObjectiveTo develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects).Design, setting, and participantsSingle-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery.ExposuresWe assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index.Main outcomes and measuresReceiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity.ResultsOf the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6%) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3% patients as "frail" with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7% as "frail."Conclusions and relevanceThe RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.

Project description:BackgroundPoor uptake and understanding of critical perioperative information represent a major safety risk for surgical patients. Implementing a patient-driven surgical safety checklist might enhance the way critical information is given and increase patient involvement in their own safety throughout the surgical pathway. The aim of this study was to develop and validate a Surgical Patient Safety Checklist (PASC) for use by surgical patients.MethodThis was a prospective study, involving patient representatives, multidisciplinary healthcare professionals and elective surgical patients to develop and validate PASC using consensus-building techniques in two Norwegian hospitals. A set of items intended for PASC were rated by patients and then submitted to Content Validation Index (CVI) analyses. Items of low CVI went through a Healthcare Failure Mode and Effect Analysis (HFMEA) Hazard Scoring process, as well as a consensus process before they were either kept or discarded. Reliability of patients' PASC ratings was assessed using Intraclass Correlation Coefficient analysis. Lastly, the face validity of PASC was investigated through focus group interviews with postoperative patients.ResultsInitial development of PASC resulted in a checklist consisting of two parts, one before (32 items) and one after surgery (26 items). After achieving consensus on the PASC content, 215 surgical patients from six surgical wards rated the items for the CVI analysis on a 1-4 scale and mostly agreed on the content. Five items were removed from the checklist, and six items were redesigned to improve PASCs' user-friendliness. The total Scale-level index/Average (S-CVI/Ave) before revision was 0.83 and 0.86 for pre- and post-operative PASC items, respectively. Following revision, these increased to 0.86 and 0.93, respectively. The PASC items reliability score was 0.97 (95% confidence interval 0.96 to 0.98). The qualitative assessment identified that patients who used PASC felt more in control of their situation; this was achieved when PASC was given to them at what they felt was the right time and healthcare professionals took part in its usage.ConclusionMultidisciplinary perioperative care staff and surgical patients agreed upon PASC content, the checklist ratings were reliable, and qualitative assessment suggested good face validity. PASC appears to be a usable and valid checklist for elective surgical patients across specialties.

Project description:BackgroundSurgical site infection (SSI) is one of the most common types of health care-associated infections. It increases mortality, prolongs hospital length of stay, and raises health care costs. Many institutions developed risk assessment models for SSI to help surgeons preoperatively identify high-risk patients and guide clinical intervention. However, most of these models had low accuracies.ObjectiveWe aimed to provide a solution in the form of an Artificial intelligence-based Multimodal Risk Assessment Model for Surgical site infection (AMRAMS) for inpatients undergoing operations, using routinely collected clinical data. We internally and externally validated the discriminations of the models, which combined various machine learning and natural language processing techniques, and compared them with the National Nosocomial Infections Surveillance (NNIS) risk index.MethodsWe retrieved inpatient records between January 1, 2014, and June 30, 2019, from the electronic medical record (EMR) system of Rui Jin Hospital, Luwan Branch, Shanghai, China. We used data from before July 1, 2018, as the development set for internal validation and the remaining data as the test set for external validation. We included patient demographics, preoperative lab results, and free-text preoperative notes as our features. We used word-embedding techniques to encode text information, and we trained the LASSO (least absolute shrinkage and selection operator) model, random forest model, gradient boosting decision tree (GBDT) model, convolutional neural network (CNN) model, and self-attention network model using the combined data. Surgeons manually scored the NNIS risk index values.ResultsFor internal bootstrapping validation, CNN yielded the highest mean area under the receiver operating characteristic curve (AUROC) of 0.889 (95% CI 0.886-0.892), and the paired-sample t test revealed statistically significant advantages as compared with other models (P<.001). The self-attention network yielded the second-highest mean AUROC of 0.882 (95% CI 0.878-0.886), but the AUROC was only numerically higher than the AUROC of the third-best model, GBDT with text embeddings (mean AUROC 0.881, 95% CI 0.878-0.884, P=.47). The AUROCs of LASSO, random forest, and GBDT models using text embeddings were statistically higher than the AUROCs of models not using text embeddings (P<.001). For external validation, the self-attention network yielded the highest AUROC of 0.879. CNN was the second-best model (AUROC 0.878), and GBDT with text embeddings was the third-best model (AUROC 0.872). The NNIS risk index scored by surgeons had an AUROC of 0.651.ConclusionsOur AMRAMS based on EMR data and deep learning methods-CNN and self-attention network-had significant advantages in terms of accuracy compared with other conventional machine learning methods and the NNIS risk index. Moreover, the semantic embeddings of preoperative notes improved the model performance further. Our models could replace the NNIS risk index to provide personalized guidance for the preoperative intervention of SSIs. Through this case, we offered an easy-to-implement solution for building multimodal RAMs for other similar scenarios.

Dataset Information

Development and validation of the Surgical Outcome Risk Tool (SORT).

Background

Methods

Results

Publications

Development and validation of the Surgical Outcome Risk Tool (SORT).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets