Dataset Information

Validating performance of TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.

ABSTRACT:

Introduction

Anatomic injury, physiological derangement, age, injury mechanism and pre-injury comorbidity are well-founded predictors of trauma outcome. Statistical prediction models may have poorer discrimination, calibration and accuracy when applied in new locations. We aimed to compare the TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.

Methods

Consecutive patients admitted to Oslo University Hospital Ullevål within 24 h after injury, with Injury Severity Score ? 10, proximal penetrating injuries, or received by trauma team, were studied. Original NORMIT coefficients were updated in a derivation dataset (NORMIT 2; n = 5923; 2005-2009). TRISS, TARN and NORMIT prediction models were evaluated in the validation dataset (n = 6348; 2010-2013) using two different AIS editions for injury coding. Exclusion due to missing data was 0.26%. Outcome was 30-day mortality. Validation included AUROC, scaled Brier statistics, and calibration plots.

Results

The NORMIT models had significantly better discrimination, calibration, and overall fit than the TRISS 09, TARN 09 and TARN 12 models. The updated NORMIT 2 had higher numerical values of AUROC and scaled Brier than the original NORMIT, but with overlapping 95%CI. Overlapping 95%CI for AUROCs and Discrimination slopes indicated that the TARN and TRISS models performed similarly. Calibration plots showed tight and consistent predictions over all Ps strata for NORMIT 2 run on AIS'98 coded data, and only little deterioration when AIS'08 data was substituted.

Conclusions

In a Norwegian trauma population, the updated Norwegian survival prediction model in trauma (NORMIT 2) performed better than well-established British and US alternatives. External validation of these three models in other Nordic populations is warranted.

SUBMITTER: Skaga NO

PROVIDER: S-EPMC5813212 | biostudies-literature | 2018 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validating performance of TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.

Skaga N O NO Eken T T Søvik S S

Acta anaesthesiologica Scandinavica 20171108 2

<h4>Introduction</h4>Anatomic injury, physiological derangement, age, injury mechanism and pre-injury comorbidity are well-founded predictors of trauma outcome. Statistical prediction models may have poorer discrimination, calibration and accuracy when applied in new locations. We aimed to compare the TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.<h4>Methods</h4>Consecutive patients admitted to Oslo University Hospital Ullevål within 24 h after injury, with I ...[more]

PMID: 29119562

Similar Datasets

Project description:BACKGROUND: Cancer survival studies are commonly analyzed using survival-time prediction models for cancer prognosis. A number of different performance metrics are used to ascertain the concordance between the predicted risk score of each patient and the actual survival time, but these metrics can sometimes conflict. Alternatively, patients are sometimes divided into two classes according to a survival-time threshold, and binary classifiers are applied to predict each patient's class. Although this approach has several drawbacks, it does provide natural performance metrics such as positive and negative predictive values to enable unambiguous assessments. METHODS: We compare the survival-time prediction and survival-time threshold approaches to analyzing cancer survival studies. We review and compare common performance metrics for the two approaches. We present new randomization tests and cross-validation methods to enable unambiguous statistical inferences for several performance metrics used with the survival-time prediction approach. We consider five survival prediction models consisting of one clinical model, two gene expression models, and two models from combinations of clinical and gene expression models. RESULTS: A public breast cancer dataset was used to compare several performance metrics using five prediction models. 1) For some prediction models, the hazard ratio from fitting a Cox proportional hazards model was significant, but the two-group comparison was insignificant, and vice versa. 2) The randomization test and cross-validation were generally consistent with the p-values obtained from the standard performance metrics. 3) Binary classifiers highly depended on how the risk groups were defined; a slight change of the survival threshold for assignment of classes led to very different prediction results. CONCLUSIONS: 1) Different performance metrics for evaluation of a survival prediction model may give different conclusions in its discriminatory ability. 2) Evaluation using a high-risk versus low-risk group comparison depends on the selected risk-score threshold; a plot of p-values from all possible thresholds can show the sensitivity of the threshold selection. 3) A randomization test of the significance of Somers' rank correlation can be used for further evaluation of performance of a prediction model. 4) The cross-validated power of survival prediction models decreases as the training and test sets become less balanced.

Project description:BackgroundPrecise models are necessary to estimate mortality risk following traumatic injury to inform clinical decision making or quantify hospital performance. The Trauma and Injury Severity Score (TRISS) has been the historical gold standard in survival prediction but its limitations are well-characterized. The present study used International Classification of Diseases 10th Revision (ICD-10) injury codes with machine learning approaches to develop models whose performance was compared to that of TRISS.MethodsThe 2015-2017 National Trauma Data Bank was used to identify patients following trauma-related admission. Injury codes from ICD-10 were grouped by clinical relevance into 1,495 variables. The TRISS score, which comprises the Injury Severity Score, age, mechanism (blunt vs penetrating) as well as highest 24-hour values for systolic blood pressure (SBP), respiratory rate (RR) and Glasgow Coma Scale (GCS) was calculated for each patient. A base eXtreme gradient boosting model (XGBoost), a machine learning technique, was developed using injury variables as well as age, SBP, RR, mechanism and GCS. Prediction of in-hospital survival and other in-hospital complications were compared between both models using receiver operating characteristic (ROC) and reliability plots. A complete XGBoost model, containing injury variables, vitals, demographic information and comorbidities, was additionally developed.ResultsOf 1,380,740 patients, 1,338,417 (96.9%) survived to discharge. Compared to survivors, those who died were older and had a greater prevalence of penetrating injuries (18.0% vs 9.44%). The base XGBoost model demonstrated a greater receiver-operating characteristic (ROC) than TRISS (0.950 vs 0.907) which persisted across sub-populations and secondary endpoints. Furthermore, it exhibited high calibration across all risk levels (R2 = 0.998 vs 0.816). The complete XGBoost model had an exceptional ROC of 0.960.ConclusionsWe report improved performance of machine learning models over TRISS. Our model may improve stratification of injury severity in clinical and quality improvement settings.

Project description:BackgroundRapid identification of high-risk polytrauma patients is crucial for early intervention and improved outcomes. This study aimed to develop and validate machine learning models for predicting 72 h mortality in adult polytrauma patients using readily available clinical parameters.MethodsA retrospective analysis was conducted on polytrauma patients from the Dryad database and our institution. Missing values pertinent to eligible individuals within the Dryad database were compensated for through the k-nearest neighbor algorithm, subsequently randomizing them into training and internal validation factions on a 7:3 ratio. The patients of our institution functioned as external validation cohorts. The predictive efficacy of random forest (RF), neural network, and XGBoost models was assessed through an exhaustive suite of performance indicators. The SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) methods were engaged to explain the supreme-performing model. Conclusively, restricted cubic spline analysis and multivariate logistic regression were employed as sensitivity analyses to verify the robustness of the findings.ResultsParameters including age, body mass index, Glasgow Coma Scale, Injury Severity Score, pH, base excess, and lactate emerged as pivotal predictors of 72 h mortality. The RF model exhibited unparalleled performance, boasting an area under the receiver operating characteristic curve (AUROC) of 0.87 (95% confidence interval [CI] 0.84-0.89), an area under the precision-recall curve (AUPRC) of 0.67 (95% CI 0.61-0.73), and an accuracy of 0.83 (95% CI 0.81-0.86) in the internal validation cohort, paralleled by an AUROC of 0.98 (95% CI 0.97-0.99), an AUPRC of 0.88 (95% CI 0.83-0.93), and an accuracy of 0.97 (95% CI 0.96-0.98) in the external validation cohort. It provided the highest net benefit in the decision curve analysis in relation to the other models. The outcomes of the sensitivity examinations were congruent with those inferred from SHAP and LIME.ConclusionsThe RF model exhibited the best performance in predicting 72 h mortality in adult polytrauma patients and has the potential to aid clinicians in identifying high-risk patients and guiding clinical decision-making.

Dataset Information

Validating performance of TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.

Introduction

Methods

Results

Conclusions

Publications

Validating performance of TRISS, TARN and NORMIT survival prediction models in a Norwegian trauma population.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets