Dataset Information

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis.

ABSTRACT:

Background

Sepsis is a life-threatening condition caused by an abnormal response of the body to infection and imposes a significant health and economic burden worldwide due to its high mortality rate. Early recognition of sepsis is crucial for effective treatment. This study aimed to systematically evaluate the performance of various machine learning models in predicting the onset of sepsis.

Methods

We conducted a comprehensive search of the Cochrane Library, PubMed, Embase, and Web of Science databases, covering studies from database inception to November 14, 2022. We used the PROBAST tool to assess the risk of bias. We calculated the predictive performance for sepsis onset using the C-index and accuracy. We followed the PRISMA guidelines for this study.

Results

We included 23 eligible studies with a total of 4,314,145 patients and 26 different machine learning models. The most frequently used models in the studies were random forest (n = 9), extreme gradient boost (n = 7), and logistic regression (n = 6) models. The random forest (test set n = 9, acc = 0.911) and extreme gradient boost (test set n = 7, acc = 0.957) models were the most accurate based on our analysis of the predictive performance. In terms of the C-index outcome, the random forest (n = 6, acc = 0.79) and extreme gradient boost (n = 7, acc = 0.83) models showed the highest performance.

Conclusion

Machine learning has proven to be an effective tool for predicting sepsis at an early stage. However, to obtain more accurate results, additional machine learning methods are needed. In our research, we discovered that the XGBoost and random forest models exhibited the best predictive performance and were most frequently utilized for predicting the onset of sepsis.

Trial registration

CRD42022384015.

SUBMITTER: Yang Z

PROVIDER: S-EPMC10523763 | biostudies-literature | 2023 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis.

Yang Zhenyu Z Cui Xiaoju X Song Zhe Z

BMC infectious diseases 20230927 1

<h4>Background</h4>Sepsis is a life-threatening condition caused by an abnormal response of the body to infection and imposes a significant health and economic burden worldwide due to its high mortality rate. Early recognition of sepsis is crucial for effective treatment. This study aimed to systematically evaluate the performance of various machine learning models in predicting the onset of sepsis.<h4>Methods</h4>We conducted a comprehensive search of the Cochrane Library, PubMed, Embase, and W ...[more]

PMID: 37759175

Similar Datasets

Project description:Background: Sepsis is among the leading causes of death in intensive care units (ICUs) worldwide and its recognition, particularly in the early stages of the disease, remains a medical challenge. The advent of an affluence of available digital health data has created a setting in which machine learning can be used for digital biomarker discovery, with the ultimate goal to advance the early recognition of sepsis. Objective: To systematically review and evaluate studies employing machine learning for the prediction of sepsis in the ICU. Data Sources: Using Embase, Google Scholar, PubMed/Medline, Scopus, and Web of Science, we systematically searched the existing literature for machine learning-driven sepsis onset prediction for patients in the ICU. Study Eligibility Criteria: All peer-reviewed articles using machine learning for the prediction of sepsis onset in adult ICU patients were included. Studies focusing on patient populations outside the ICU were excluded. Study Appraisal and Synthesis Methods: A systematic review was performed according to the PRISMA guidelines. Moreover, a quality assessment of all eligible studies was performed. Results: Out of 974 identified articles, 22 and 21 met the criteria to be included in the systematic review and quality assessment, respectively. A multitude of machine learning algorithms were applied to refine the early prediction of sepsis. The quality of the studies ranged from "poor" (satisfying ≤ 40% of the quality criteria) to "very good" (satisfying ≥ 90% of the quality criteria). The majority of the studies (n = 19, 86.4%) employed an offline training scenario combined with a horizon evaluation, while two studies implemented an online scenario (n = 2, 9.1%). The massive inter-study heterogeneity in terms of model development, sepsis definition, prediction time windows, and outcomes precluded a meta-analysis. Last, only two studies provided publicly accessible source code and data sources fostering reproducibility. Limitations: Articles were only eligible for inclusion when employing machine learning algorithms for the prediction of sepsis onset in the ICU. This restriction led to the exclusion of studies focusing on the prediction of septic shock, sepsis-related mortality, and patient populations outside the ICU. Conclusions and Key Findings: A growing number of studies employs machine learning to optimize the early prediction of sepsis through digital biomarker discovery. This review, however, highlights several shortcomings of the current approaches, including low comparability and reproducibility. Finally, we gather recommendations how these challenges can be addressed before deploying these models in prospective analyses. Systematic Review Registration Number: CRD42020200133.

Project description:IntroductionApproximately 7,000 newborns die every day, accounting for almost half of child deaths under 5 years of age. Deciphering which neonates are at increased risk for mortality can have an important global impact. As such, integrating high computational technology (e.g., artificial intelligence [AI]) may help identify the early and potentially modifiable predictors of neonatal mortality. Therefore, the objective of this study was to collate, critically appraise, and analyze neonatal prediction studies that included AI.MethodsA literature search was performed in PubMed, Cochrane, OVID, and Google Scholar. We included studies that used AI (e.g., machine learning (ML) and deep learning) to formulate prediction models for neonatal death. We excluded small studies (n < 500 individuals) and studies using only antenatal factors to predict mortality. Two independent investigators screened all articles for inclusion. The data collection consisted of study design, number of models, features used per model, feature importance, internal and/or external validation, and calibration analysis. Our primary outcome was the average area under the receiving characteristic curve (AUC) or sensitivity and specificity for all models included in each study.ResultsOf 434 articles, 11 studies were included. The total number of participants was 1.26 M with gestational ages ranging from 22 weeks to term. Number of features ranged from 3 to 66 with timing of prediction as early as 5 min of life to a maximum of 7 days of age. The average number of models per study was 4, with neural network, random forest, and logistic regression comprising the most used models (58.3%). Five studies (45.5%) reported calibration plots and 2 (18.2%) conducted external validation. Eight studies reported results by AUC and 5 studies reported the sensitivity and specificity. The AUC varied from 58.3% to 97.0%. The mean sensitivities ranged from 63% to 80% and specificities from 78% to 99%. The best overall model was linear discriminant analysis, but it also had a high number of features (n = 17).Discussion/conclusionML models can accurately predict death in neonates. This analysis demonstrates the most commonly used predictors and metrics for AI prediction models for neonatal mortality. Future studies should focus on external validation, calibration, as well as deployment of applications that can be readily accessible to health-care providers.

Project description:INTRODUCTION: To systematically review studies evaluating the performance of Sequential Organ Failure Assessment (SOFA)-based models for predicting mortality in patients in the intensive care unit (ICU). METHODS: Medline, EMBASE and other databases were searched for English-language articles with the major objective of evaluating the prognostic performance of SOFA-based models in predicting mortality in surgical and/or medical ICU admissions. The quality of each study was assessed based on a quality framework for prognostic models. RESULTS: Eighteen articles met all inclusion criteria. The studies differed widely in the SOFA derivatives used and in their methods of evaluation. Ten studies reported about developing a probabilistic prognostic model, only five of which used an independent validation data set. The other studies used the SOFA-based score directly to discriminate between survivors and non-survivors without fitting a probabilistic model. In five of the six studies, admission-based models (Acute Physiology and Chronic Health Evaluation (APACHE) II/III) were reported to have a slightly better discrimination ability than SOFA-based models at admission (the receiver operating characteristic curve (AUC) of SOFA-based models ranged between 0.61 and 0.88), and in one study a SOFA model had higher AUC than the Simplified Acute Physiology Score (SAPS) II model. Four of these studies used the Hosmer-Lemeshow tests for calibration, none of which reported a lack of fit for the SOFA models. Models based on sequential SOFA scores were described in 11 studies including maximum SOFA scores and maximum sum of individual components of the SOFA score (AUC range: 0.69 to 0.92) and delta SOFA (AUC range: 0.51 to 0.83). Studies comparing SOFA with other organ failure scores did not consistently show superiority of one scoring system to another. Four studies combined SOFA-based derivatives with admission severity of illness scores, and they all reported on improved predictions for the combination. Quality of studies ranged from 11.5 to 19.5 points on a 20-point scale. CONCLUSIONS: Models based on SOFA scores at admission had only slightly worse performance than APACHE II/III and were competitive with SAPS II models in predicting mortality in patients in the general medical and/or surgical ICU. Models with sequential SOFA scores seem to have a comparable performance with other organ failure scores. The combination of sequential SOFA derivatives with APACHE II/III and SAPS II models clearly improved prognostic performance of either model alone. Due to the heterogeneity of the studies, it is impossible to draw general conclusions on the optimal mathematical model and optimal derivatives of SOFA scores. Future studies should use a standard evaluation methodology with a standard set of outcome measures covering discrimination, calibration and accuracy.

Project description:BackgroundHaemorrhagic transformation (HT) is a severe complication after ischaemic stroke, but identifying patients at high risks remains challenging. Although numerous prediction models have been developed for HT following thrombolysis, thrombectomy, or spontaneous occurrence, a comprehensive summary is lacking. This study aimed to review and compare traditional and machine learning-based HT prediction models, focusing on their development, validation, and diagnostic accuracy.MethodsPubMed and Ovid-Embase were searched for observational studies or randomised controlled trials related to traditional or machine learning-based models. Data were extracted according to Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) checklist and risk of bias was assessed using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). Performance data for prediction models that were externally validated at least twice and showed low risk of bias were meta-analysed.ResultsA total of 100 studies were included, with 67 focusing on model development and 33 on model validation. Among 67 model development studies, 44 were traditional model studies involving 47 prediction models (with National Institutes of Health Stroke Scale score being the most frequently used predictor in 35 models), and 23 studies focused on machine learning prediction models (with support vector machines being the most common algorithm, used in 10 models). The 33 validation studies externally validated 34 traditional prediction models. Regarding study quality, 26 studies were assessed as having a low risk of bias, 11 as unclear, and 63 as high risk of bias. Meta-analysis of 15 studies validating eight models showed a pooled area under the receiver operating characteristic curve of approximately 0.70 for predicting HT.ConclusionWhile significant progress has been made in developing HT prediction models, both traditional and machine learning-based models still have limitations in methodological rigour, predictive accuracy, and clinical applicability. Future models should undergo more rigorous validation, adhere to standardised reporting frameworks, and prioritise predictors that are both statistically significant and clinically meaningful. Collaborative efforts across research groups are essential for validating these models in diverse populations and improving their broader applicability in clinical practice.Systematic review registrationInternational Prospective Register of Systematic Reviews (CRD42022332816).

Dataset Information

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis.

Background

Methods

Results

Conclusion

Trial registration

Publications

Predicting sepsis onset in ICU using machine learning models: a systematic review and meta-analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets