Dataset Information

The prognostic accuracy evaluation of SAPS 3, SOFA and APACHE II scores for mortality prediction in the surgical ICU: an external validation study and decision-making analysis.

ABSTRACT: BACKGROUND:The early postoperative period is critical for surgical patients. SOFA, SAPS 3 and APACHE II are prognostic scores widely used to predict mortality in ICU patients. This study aimed to evaluate these index tests for their prognostic accuracy for intra-ICU and in-hospital mortalities as target conditions in patients admitted to ICU after urgent or elective surgeries and to test whether they aid in decision-making. The process comprised the assessment of discrimination through analysis of the areas under the receiver operating characteristic curves and calibration of the prognostic models for the target conditions. After, the clinical relevance of applying them was evaluated through the measurement of the net benefit of their use in the clinical decision. RESULTS:Index tests were found to discriminate regular for both target conditions with a poor calibration (C statistics-intra-ICU mortality AUROCs: APACHE II 0.808, SAPS 3 0.821 and SOFA 0.797/in-hospital mortality AUROCs: APACHE II 0.772, SAPS 3 0.790 and SOFA 0.742). Calibration assessment revealed a weak correlation between the observed and expected number of cases in several thresholds of risk, calculated by each model, for both tested outcomes. The net benefit analysis showed that all score's aggregate value in the clinical decision when the calculated probabilities of death ranged between 10 and 40%. CONCLUSIONS:In this study, we observed that the tested ICU prognostic scores are fair tools for intra-ICU and in-hospital mortality prediction in a cohort of postoperative surgical patients. Also, they may have some potential to be used as ancillary data to support decision-making by physicians and families regarding the level of therapeutic investment and palliative care.

SUBMITTER: Falcao ALE

PROVIDER: S-EPMC6353976 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:The Sequential Organ Failure Assessment (SOFA) score is commonly used in ICUs around the world, designed to assess the severity of the patient's clinical state based on function/dysfunction of six major organ systems. The goal of this work is to build a computational model to predict mortality based on a series of SOFA scores. In addition, we examined the possibility of improving the prediction by incorporating a new component designed to measure the performance of the gastrointestinal system, added to the other six components. METHODS:In this retrospective study, we used patients' three latest SOFA scores recorded during an individual ICU stay as input to different machine learning models and ensemble learning models. We added three validated parameters representing gastrointestinal failure. Among others, we used classification models such as Support Vector Machines (SVMs), Neural Networks, Logistic Regression and a penalty function used to increase model robustness in regard to certain extreme cases, which may be found in ICU population. We used the Area under Curve (AUC) performance metric to examine performance. RESULTS:We found an ensemble model of linear and logistic regression achieves a higher AUC compared related works in past years. After incorporating the gastrointestinal failure score along with the penalty function, our best performing ensemble model resulted in an additional improvement in terms of AUC metrics. We implemented and compared 36 different models that were built using both the information from the SOFA score as well as that of the gastrointestinal system. All compared models have approximately similar and relatively large AUC (between 0.8645 and 0.9146) with the best results are achieved by incorporating the gastrointestinal parameters into the prediction models. CONCLUSIONS:Our findings indicate that gastrointestinal parameters carry significant information as a mortality predictor in addition to the conventional SOFA score. This information improves the predictive power of machine learning models by extending the SOFA to include information related to gastrointestinal organ system. The described method improves mortality prediction by considering the dynamics of the extended SOFA score. Although tested on a limited data set, the results' stability across different models suggests robustness in real-time use.

Project description:BACKGROUND:The performance of severity-of-illness scores varies in different scenarios and must be validated prior of being used in a specific settings and geographic regions. Moreover, models' calibration may deteriorate overtime and performance of such instruments should be reassessed regularly. Therefore, we aimed at to validate the SAPS 3 in a large contemporary cohort of patients admitted to Brazilian ICUs. In addition, we also compared the performance of the SAPS 3 with the MPM0-III. METHODS:This is a retrospective cohort study in which 48,816 (medical admissions = 67.9%) adult patients are admitted to 72 Brazilian ICUs during 2013. We evaluated models' discrimination using the area under the receiver operating characteristic curve (AUROC). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration). RESULTS:Mean SAPS 3 score was 44.3 ± 15.4 points. ICU and hospital mortality rates were 11.0 and 16.5%. We estimated predicted mortality using both standard (SE) and Central and South American (CSA) customized equations. Predicted mortality rates were 16.4 ± 19.3% (SAPS 3-SE), 21.7 ± 23.2% (SAPS 3-CSA) and 14.3 ± 14.0% (MPM0-III). Standardized mortality ratios (SMR) obtained for each model were: 1.00 (95% CI, 0.98-0.102) for the SAPS 3-SE, 0.75 (0.74-0.77) for the SAPS 3-CSA and 1.15 (1.13-1.18) for the MPM0-III. Discrimination was better for SAPS 3 models (AUROC = 0.85) than for MPM0-III (AUROC = 0.80) (p < 0.001). We applied the calibration belt to evaluate the agreement between observed and expected mortality rates (calibration): the SAPS 3-CSA overestimated mortality throughout all risk classes while the MPM0-III underestimated it uniformly. The SAPS 3-SE did not show relevant deviations from ideal calibration. CONCLUSIONS:In a large contemporary database, the SAPS 3-SE was accurate in predicting outcomes, supporting its use for performance evaluation and benchmarking in Brazilian ICUs.

Project description:BackgroundSurgical patients incur preventable harm from cognitive and judgment errors made under time constraints and uncertainty regarding patients' diagnoses and predicted response to treatment. Decision analysis and techniques of reinforcement learning theoretically can mitigate these challenges but are poorly understood and rarely used clinically. This review seeks to promote an understanding of decision analysis and reinforcement learning by describing their use in the context of surgical decision-making.MethodsCochrane, EMBASE, and PubMed databases were searched from their inception to June 2019. Included were 41 articles about cognitive and diagnostic errors, decision-making, decision analysis, and machine-learning. The articles were assimilated into relevant categories according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines.ResultsRequirements for time-consuming manual data entry and crude representations of individual patients and clinical context compromise many traditional decision-support tools. Decision analysis methods for calculating probability thresholds can inform population-based recommendations that jointly consider risks, benefits, costs, and patient values but lack precision for individual patient-centered decisions. Reinforcement learning, a machine-learning method that mimics human learning, can use a large set of patient-specific input data to identify actions yielding the greatest probability of achieving a goal. This methodology follows a sequence of events with uncertain conditions, offering potential advantages for personalized, patient-centered decision-making. Clinical application would require secure integration of multiple data sources and attention to ethical considerations regarding liability for errors and individual patient preferences.ConclusionTraditional decision-support tools are ill-equipped to accommodate time constraints and uncertainty regarding diagnoses and the predicted response to treatment, both of which often impair surgical decision-making. Decision analysis and reinforcement learning have the potential to play complementary roles in delivering high-value surgical care through sound judgment and optimal decision-making.

Dataset Information

The prognostic accuracy evaluation of SAPS 3, SOFA and APACHE II scores for mortality prediction in the surgical ICU: an external validation study and decision-making analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets