Dataset Information

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy.

ABSTRACT:

Purpose

Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis.

Methods

A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learning model for real-time prediction of these conditions. Quality of evidence was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) methodology, with a tailored Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist to evaluate risk of bias. Models with a reported area under the curve of the receiver operating characteristic (AUROC) metric were meta-analyzed to identify strongest contributors to model performance.

Results

After screening, a total of 28 papers were eligible for synthesis, from which 130 models were extracted. The majority of papers were developed in the intensive care unit (ICU, n?=?15; 54%), followed by hospital wards (n?=?7; 25%), the emergency department (ED, n?=?4; 14%) and all of these settings (n?=?2; 7%). For the prediction of sepsis, diagnostic test accuracy assessed by the AUROC ranged from 0.68-0.99 in the ICU, to 0.96-0.98 in-hospital and 0.87 to 0.97 in the ED. Varying sepsis definitions limit pooling of the performance across studies. Only three papers clinically implemented models with mixed results. In the multivariate analysis, temperature, lab values, and model type contributed most to model performance.

Conclusion

This systematic review and meta-analysis show that on retrospective data, individual machine learning models can accurately predict sepsis onset ahead of time. Although they present alternatives to traditional scoring systems, between-study heterogeneity limits the assessment of pooled results. Systematic reporting and clinical implementation studies are needed to bridge the gap between bytes and bedside.

SUBMITTER: Fleuren LM

PROVIDER: S-EPMC7067741 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy.

Fleuren Lucas M LM Klausch Thomas L T TLT Zwager Charlotte L CL Schoonmade Linda J LJ Guo Tingjie T Roggeveen Luca F LF Swart Eleonora L EL Girbes Armand R J ARJ Thoral Patrick P Ercole Ari A Hoogendoorn Mark M Elbers Paul W G PWG

Intensive care medicine 20200121 3

<h4>Purpose</h4>Early clinical recognition of sepsis can be challenging. With the advancement of machine learning, promising real-time models to predict sepsis have emerged. We assessed their performance by carrying out a systematic review and meta-analysis.<h4>Methods</h4>A systematic search was performed in PubMed, Embase.com and Scopus. Studies targeting sepsis, severe sepsis or septic shock in any hospital setting were eligible for inclusion. The index test was any supervised machine learnin ...[more]

PMID: 31965266

Similar Datasets

Project description:Background and objectivesSepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points.MethodsPubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction.ResultsFifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools.ConclusionMachine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.

Project description:BackgroundSystematic review is an indispensable tool for optimal evidence collection and evaluation in evidence-based medicine. However, the explosive increase of the original literatures makes it difficult to accomplish critical appraisal and regular update. Artificial intelligence (AI) algorithms have been applied to automate the literature screening procedure in medical systematic reviews. In these studies, different algorithms were used and results with great variance were reported. It is therefore imperative to systematically review and analyse the developed automatic methods for literature screening and their effectiveness reported in current studies.MethodsAn electronic search will be conducted using PubMed, Embase, ACM Digital Library, and IEEE Xplore Digital Library databases, as well as literatures found through supplementary search in Google scholar, on automatic methods for literature screening in systematic reviews. Two reviewers will independently conduct the primary screening of the articles and data extraction, in which nonconformities will be solved by discussion with a methodologist. Data will be extracted from eligible studies, including the basic characteristics of study, the information of training set and validation set, and the function and performance of AI algorithms, and summarised in a table. The risk of bias and applicability of the eligible studies will be assessed by the two reviewers independently based on Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2). Quantitative analyses, if appropriate, will also be performed.DiscussionAutomating systematic review process is of great help in reducing workload in evidence-based practice. Results from this systematic review will provide essential summary of the current development of AI algorithms for automatic literature screening in medical evidence synthesis and help to inspire further studies in this field.Systematic review registrationPROSPERO CRD42020170815 (28 April 2020).

Project description:Background: Sepsis is among the leading causes of death in intensive care units (ICUs) worldwide and its recognition, particularly in the early stages of the disease, remains a medical challenge. The advent of an affluence of available digital health data has created a setting in which machine learning can be used for digital biomarker discovery, with the ultimate goal to advance the early recognition of sepsis. Objective: To systematically review and evaluate studies employing machine learning for the prediction of sepsis in the ICU. Data Sources: Using Embase, Google Scholar, PubMed/Medline, Scopus, and Web of Science, we systematically searched the existing literature for machine learning-driven sepsis onset prediction for patients in the ICU. Study Eligibility Criteria: All peer-reviewed articles using machine learning for the prediction of sepsis onset in adult ICU patients were included. Studies focusing on patient populations outside the ICU were excluded. Study Appraisal and Synthesis Methods: A systematic review was performed according to the PRISMA guidelines. Moreover, a quality assessment of all eligible studies was performed. Results: Out of 974 identified articles, 22 and 21 met the criteria to be included in the systematic review and quality assessment, respectively. A multitude of machine learning algorithms were applied to refine the early prediction of sepsis. The quality of the studies ranged from "poor" (satisfying ≤ 40% of the quality criteria) to "very good" (satisfying ≥ 90% of the quality criteria). The majority of the studies (n = 19, 86.4%) employed an offline training scenario combined with a horizon evaluation, while two studies implemented an online scenario (n = 2, 9.1%). The massive inter-study heterogeneity in terms of model development, sepsis definition, prediction time windows, and outcomes precluded a meta-analysis. Last, only two studies provided publicly accessible source code and data sources fostering reproducibility. Limitations: Articles were only eligible for inclusion when employing machine learning algorithms for the prediction of sepsis onset in the ICU. This restriction led to the exclusion of studies focusing on the prediction of septic shock, sepsis-related mortality, and patient populations outside the ICU. Conclusions and Key Findings: A growing number of studies employs machine learning to optimize the early prediction of sepsis through digital biomarker discovery. This review, however, highlights several shortcomings of the current approaches, including low comparability and reproducibility. Finally, we gather recommendations how these challenges can be addressed before deploying these models in prospective analyses. Systematic Review Registration Number: CRD42020200133.

Project description:Muscle ultrasound is an emerging tool for diagnosing sarcopenia. This review aims to summarize the current knowledge on the diagnostic test accuracy of ultrasound for the diagnosis of sarcopenia. We collected data from Ovid Medline, Embase and the Cochrane Central Register of Controlled Trials. Diagnostic test accuracy studies using muscle ultrasound to detect sarcopenia were included. Bivariate random-effects models based on sensitivity and specificity pairs were used to calculate the pooled estimates of sensitivity, specificity and the area under the curves (AUCs) of summary receiver operating characteristic (SROC), if possible. We screened 7332 publications and included 17 studies with 2143 participants (mean age range: 52.6-82.8 years). All included studies had a high risk of bias. The study populations, reference standards and ultrasound measurement methods varied across the studies. Lower extremity muscles were commonly studied, whereas muscle thickness (MT) was the most widely measured parameter, followed by the cross-sectional area (CSA). The MTs of the gastrocnemius, rectus femoris, tibialis anterior, soleus, rectus abdominis and geniohyoid muscles showed a moderate diagnostic accuracy for sarcopenia (SROC-AUC 0.83, 8 studies; SROC-AUC 0.78, 5 studies; AUC 0.82, 1 study; AUC 0.76-0.78, 2 studies; AUC 0.76, 1 study; and AUC 0.79, 1 study, respectively), whereas the MTs of vastus intermedius, quadriceps femoris and transversus abdominis muscles showed a low diagnostic accuracy (AUC 0.67-0.71, 3 studies; SROC-AUC 0.64, 4 studies; and AUC 0.68, 1 study, respectively). The CSA of rectus femoris, biceps brachii muscles and gastrocnemius fascicle length also showed a moderate diagnostic accuracy (AUC 0.70-0.90, 3 studies; 0.81, 1 study; and 0.78-0.80, 1 study, respectively), whereas the echo intensity (EI) of rectus femoris, vastus intermedius, quadriceps femoris and biceps brachii muscles showed a low diagnostic accuracy (AUC 0.52-0.67, 2 studies; 0.48-0.50, 1 study; 0.43-0.49, 1 study; and 0.69, 1 study, respectively). The combination of CSA and EI of biceps brachii or rectus femoris muscles was better than either CSA or EI alone for diagnosing sarcopenia. Muscle ultrasound shows a low-to-moderate diagnostic test accuracy for sarcopenia diagnosis depending on different ultrasound parameters, measured muscles, reference standards and study populations. The combination of muscle quality indicators (e.g., EI) and muscle quantity indicators (e.g., MT) might provide better diagnostic test accuracy.

Dataset Information

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy.

Purpose

Methods

Results

Conclusion

Publications

Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets