A Machine Learning Classifier Improves Mortality Prediction Compared With Pediatric Logistic Organ Dysfunction-2 Score: Model Development and Validation.
Ontology highlight
ABSTRACT: Objectives
To determine whether machine learning algorithms can better predict PICU mortality than the Pediatric Logistic Organ Dysfunction-2 score.Design
Retrospective study.Setting
Quaternary care medical-surgical PICU.Patients
All patients admitted to the PICU from 2013 to 2019.Interventions
None.Measurements and main results
We investigated the performance of various machine learning algorithms using the same variables used to calculate the Pediatric Logistic Organ Dysfunction-2 score to predict PICU mortality. We used 10,194 patient records from 2013 to 2017 for training and 4,043 patient records from 2018 to 2019 as a holdout validation cohort. Mortality rate was 3.0% in the training cohort and 3.4% in the validation cohort. The best performing algorithm was a random forest model (area under the receiver operating characteristic curve, 0.867 [95% CI, 0.863-0.895]; area under the precision-recall curve, 0.327 [95% CI, 0.246-0.414]; F1, 0.396 [95% CI, 0.321-0.468]) and significantly outperformed the Pediatric Logistic Organ Dysfunction-2 score (area under the receiver operating characteristic curve, 0.761 [95% CI, 0.713-0.810]; area under the precision-recall curve (0.239 [95% CI, 0.165-0.316]; F1, 0.284 [95% CI, 0.209-0.360]), although this difference was reduced after retraining the Pediatric Logistic Organ Dysfunction-2 logistic regression model at the study institution. The random forest model also showed better calibration than the Pediatric Logistic Organ Dysfunction-2 score, and calibration of the random forest model remained superior to the retrained Pediatric Logistic Organ Dysfunction-2 model.Conclusions
A machine learning model achieved better performance than a logistic regression-based score for predicting ICU mortality. Better estimation of mortality risk can improve our ability to adjust for severity of illness in future studies, although external validation is required before this method can be widely deployed.
SUBMITTER: Prince RD
PROVIDER: S-EPMC8133049 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Abstract Aims and objectives Utility of pediatric logistic organ dysfunction-2 (PELOD-2) score on day 1 within 1 hour of admission in predicting mortality in children admitted in pediatric intensive care unit (PICU). Background Various scoring systems aid to evaluate the patient's mortality risk in the intensive care unit (ICU) by assigning a score and predicting the outcome. Critically ill children are characterized by large variations in the normal body homeostasis. These variations can be estimated by the change of the physiological variables from the normal range. Various scores are constructed from deviations of these changed variables. One such score, the PELOD-2 score, is used to predict mortality of patients admitted in PICU. Materials and methods This study was carried out at a tertiary care center in central India to study the utility of PELOD-2 score within 1 hour of admission to predict mortality in patients admitted in PICU. Results Total 129 patients were included in this study with mean age of 67 months. The system with highest admission was central nervous system with 42 children and 16.6% mortality, whereas those 7 patients with hematological system involvement had highest mortality of 28.5%. The mortality rate was 15.55%. In our study for PELOD-2 within 24 hours of admission, the area under receiver operating curve was 0.87 and the Hosmer–Lemeshow test was p = 0.42. Conclusion Pediatric logistic organ dysfunction-2 score in our study had significant association with mortality along with the Hosmer–Lemeshow goodness-of-fit test showing a good prediction of mortality. How to cite this article Deshmukh T, Varma A, Damke S, Meshram R. Predictive Efficacy of Pediatric Logistic Organ Dysfunction-2 Score in Pediatric Intensive Care Unit of Rural Hospital. Indian J Crit Care Med 2020;24(8):701–704.
| S-EPMC7519618 | biostudies-literature
Project description:OBJECTIVES:Develop and test the performance of electronic version of the Children's Hospital of Pittsburgh Pediatric Risk of Mortality-IV and electronic version of the Children's Hospital of Pittsburgh Pediatric Logistic Organ Dysfunction-2 scores. DESIGN:Retrospective, single-center cohort derived from structured electronic health record data. SETTING:Large, quaternary PICU at a freestanding, university-affiliated children's hospital. PATIENTS:All encounters with a PICU admission between January 1, 2009, and December 31, 2017, identified using electronic definitions of inpatient encounter. INTERVENTIONS:None. MEASUREMENTS AND MAIN RESULTS:The main outcome was predictive validity of each score for hospital mortality, assessed as model discrimination and calibration. Discrimination was examined with the area under the receiver operating characteristics curve and the area under the precision-recall curve. Calibration was assessed with the Hosmer-Lemeshow goodness of fit test and calculation of a standardized mortality ratio. Models were recalibrated with new regression coefficients in a training subset of 75% of encounters selected randomly from all years of the cohort and the calibrated models were tested in the remaining 25% of the cohort. Content validity was assessed by examining correlation between electronic versions of the scores and prospectively calculated data (electronic version of the Children's Hospital of Pittsburgh Pediatric Risk of Mortality-IV) and an alternative informatics approach (Children's Hospital of Pittsburgh Pediatric Logistic Organ Dysfunction-2 score). The cohort included 21,335 encounters. Correlation coefficients indicated strong agreement between different methods of score calculation. Uncalibrated area under the receiver operating characteristics curves were 0.96 (95% CI, 0.95-0.97) for electronic version of the Children's Hospital of Pittsburgh Pediatric Logistic Organ Dysfunction-2 score and 0.87 (95% CI, 0.85-0.89) for electronic version of the Children's Hospital of Pittsburgh Pediatric Risk of Mortality-IV for inpatient mortality. The uncalibrated electronic version of the Children's Hospital of Pittsburgh Pediatric Risk of Mortality-IV standardized mortality ratio was 0.63 (0.59-0.66), demonstrating strong agreement with previous, prospective evaluation at the study center. The uncalibrated electronic version of the Children's Hospital of Pittsburgh Pediatric Logistic Organ Dysfunction-2 score standardized mortality ratio was 0.20 (0.18-0.21). All models required recalibrating (all Hosmer-Lemeshow goodness-of-fit, p < 0.001) and subsequently demonstrated acceptable goodness-of-fit when examined in a test subset (n = 5,334) of the cohort. CONCLUSIONS:Electronically derived intensive care acuity scores demonstrate very good to excellent discrimination and can be calibrated to institutional outcomes. This approach can facilitate both performance improvement and research initiatives and may offer a scalable strategy for comparison of interinstitutional PICU outcomes.
| S-EPMC7115250 | biostudies-literature
Project description:Since its introduction into the medical literature in the 1970s, the term multiple organ dysfunction syndrome (or some variant) has been applied broadly to any patient with >1 concurrent organ dysfunction. However, the epidemiology, mechanisms, time course, and outcomes among children with multiple organ dysfunction vary substantially. We posit that the term pediatric multiple organ dysfunction syndrome (or MODS) should be reserved for patients with a systemic pathologic state resulting from a common mechanism (or mechanisms) that affects numerous organ systems simultaneously. In contrast, children in whom organ injuries are attributable to distinct mechanisms should be considered to have additive organ system dysfunctions but not the syndrome of MODS. Although such differentiation may not always be possible with current scientific knowledge, we make the case for how attempts to differentiate multiple organ dysfunction from other states of additive organ dysfunctions can help to evolve clinical and research priorities in diagnosis, monitoring, and therapy from largely organ-specific to more holistic strategies.
| S-EPMC9084565 | biostudies-literature
Project description:ObjectiveTo describe the state of the science, identify knowledge gaps, and offer potential future research questions regarding promising therapies for children with multiple organ dysfunction syndrome presented during the Eunice Kennedy Shriver National Institute of Child Health and Human Development Workshop on Pediatric Multiple Organ Dysfunction Syndrome (March 26-27, 2015).Data sourcesLiterature review, research data, and expert opinion.Study selectionNot applicable.Data extractionModerated by an expert from the field, issues relevant to the association of multiple organ dysfunction syndrome with a variety of conditions were presented, discussed, and debated with a focus on identifying knowledge gaps and research priorities.Data synthesisSummary of presentations and discussion supported and supplemented by relevant literature.ConclusionsAmong critically ill children, multiple organ dysfunction syndrome is relatively common and associated with significant morbidity and mortality. For outcomes to improve, effective therapies aimed at preventing and treating this condition must be discovered and rigorously evaluated. In this article, a number of potential opportunities to enhance current care are highlighted including the need for a better understanding of the pharmacokinetics and pharmacodynamics of medications, the effect of early and optimized nutrition, and the impact of effective glucose control in the setting of multiple organ dysfunction syndrome. Additionally, a handful of the promising therapies either currently being implemented or developed are described. These include extracorporeal therapies, anticytokine therapies, antitoxin treatments, antioxidant approaches, and multiple forms of exogenous steroids. For the field to advance, promising therapies and other therapies must be assessed in rigorous manner and implemented accordingly.
| S-EPMC5333132 | biostudies-literature
Project description:OBJECTIVES:Limited data exist about the timing and significance of mitochondrial alterations in children with sepsis. We therefore sought to determine if alterations in mitochondrial respiration and content within circulating peripheral blood mononuclear cells were associated with organ dysfunction in pediatric sepsis. DESIGN:Prospective observational study SETTING:: Single academic PICU. PATIENTS:One-hundred sixty-seven children with sepsis/septic shock and 19 PICU controls without sepsis, infection, or organ dysfunction. INTERVENTIONS:None. MEASUREMENTS AND MAIN RESULTS:Mitochondrial respiration and content were measured in peripheral blood mononuclear cells on days 1-2, 3-5, and 8-14 after sepsis recognition or once for controls. Severity and duration of organ dysfunction were determined using the Pediatric Logistic Organ Dysfunction score and organ failure-free days through day 28. Day 1-2 maximal uncoupled respiration (9.7?±?7.7 vs 13.7?±?4.1 pmol O2/s/10 cells; p = 0.02) and spare respiratory capacity (an index of bioenergetic reserve: 6.2?±?4.3 vs 9.6?±?3.1; p = 0.005) were lower in sepsis than controls. Mitochondrial content, measured by mitochondrial DNA/nuclear DNA, was higher in sepsis on day 1-2 than controls (p = 0.04) and increased in sepsis patients who had improving spare respiratory capacity over time (p = 0.005). Mitochondrial respiration and content were not associated with day 1-2 Pediatric Logistic Organ Dysfunction score, but low spare respiratory capacity was associated with higher Pediatric Logistic Organ Dysfunction score on day 3-5. Persistently low spare respiratory capacity was predictive of residual organ dysfunction on day 14 (area under the receiver operating characteristic, 0.72; 95% CI, 0.61-0.84) and trended toward fewer organ failure-free days although day 28 (? coefficient, -0.64; 95% CI, -1.35 to 0.06; p = 0.08). CONCLUSIONS:Mitochondrial respiration was acutely decreased in peripheral blood mononuclear cells in pediatric sepsis despite an increase in mitochondrial content. Over time, a rise in mitochondrial DNA tracked with improved respiration. Although initial mitochondrial alterations in peripheral blood mononuclear cells were unrelated to organ dysfunction, persistently low respiration was associated with slower recovery from organ dysfunction.
| S-EPMC7341116 | biostudies-literature
Project description:ObjectiveTo summarize the epidemiology and outcomes of children with multiple organ dysfunction syndrome as part of the Eunice Kennedy Shriver National Institute of Child Health and Human Development multiple organ dysfunction syndrome workshop (March 26-27, 2015).Data sourcesLiterature review, research data, and expert opinion.Study selectionNot applicable.Data extractionModerated by an experienced expert from the field, issues relevant to the epidemiology and outcomes of children with multiple organ dysfunction syndrome were presented, discussed, and debated with a focus on identifying knowledge gaps and research priorities.Data synthesisSummary of presentations and discussion supported and supplemented by the relevant literature.ConclusionsA full understanding the epidemiology and outcome of multiple organ dysfunction syndrome in children is limited by inconsistent definitions and populations studied. Nonetheless, pediatric multiple organ dysfunction syndrome is common among PICU patients, occurring in up to 57% depending on the population studied; sepsis remains its leading cause. Pediatric multiple organ dysfunction syndrome leads to considerable short-term morbidity and mortality. Long-term outcomes of multiple organ dysfunction syndrome in children have not been well studied; however, studies of adults and children with other critical illnesses suggest that the risk of long-term adverse sequelae is high. Characterization of the long-term outcomes of pediatric multiple organ dysfunction syndrome is crucial to identify opportunities for improved treatment and recovery strategies that will improve the quality of life of critically ill children and their families. The workshop identified important knowledge gaps and research priorities intended to promote the development of standard definitions and the identification of modifiable factors related to its occurrence and outcome.
| S-EPMC5334773 | biostudies-literature
Project description:Background Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. Methods An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. Results Manual screening for substance misuse was completed in 67.8% (n?=?56,227) with 1.1% (n?=?628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99–0.99) across the encounter and 0.98 (95% CI 0.98–0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77–0.84) and 0.72 (95% CI 0.68–0.75). For the first 24 h, they were 0.75 (95% CI 0.71–0.78) and 0.61 (95% CI 0.57–0.64). Conclusions Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers. Supplementary Information The online version contains supplementary material available at 10.1186/s13722-021-00229-7.
| S-EPMC7967783 | biostudies-literature
Project description:OBJECTIVES:To assess the validity of Vasoactive-Inotropic Score as a scoring system for cardiovascular support and surrogate outcome in pediatric sepsis. DESIGN:Secondary retrospective analysis of a single-center sepsis registry. SETTING:Freestanding children's hospital and tertiary referral center. PATIENTS:Children greater than 60 days and less than 18 years with sepsis identified in the emergency department between January 2012 and June 2015 treated with at least one vasoactive medication within 48 hours of admission to the PICU. INTERVENTIONS:None. MEASUREMENTS AND MAIN RESULTS:Vasoactive-Inotropic Score was abstracted at 6, 12, 24, and 48 hours post ICU admission. Primary outcomes were ventilator days and ICU length of stay. The secondary outcome was a composite outcome of cardiac arrest/extracorporeal membrane oxygenation/in-hospital mortality. One hundred thirty-eight patients met inclusion criteria. Most common infectious sources were pneumonia (32%) and bacteremia (23%). Thirty-three percent were intubated and mortality was 6%. Of the time points assessed, Vasoactive-Inotropic Score at 48 hours showed the strongest correlation with ICU length of stay (r = 0.53; p < 0.0001) and ventilator days (r = 0.52; p < 0.0001). On multivariable analysis, Vasoactive-Inotropic Score at 48 hours was a strong independent predictor of primary outcomes and intubation. For every unit increase in Vasoactive-Inotropic Score at 48 hours, there was a 13% increase in ICU length of stay (p < 0.001) and 8% increase in ventilator days (p < 0.01). For every unit increase in Vasoactive-Inotropic Score at 12 hours, there was a 14% increase in odds of having the composite outcome (p < 0.01). CONCLUSIONS:Vasoactive-Inotropic Score in pediatric sepsis patients is independently associated with important clinically relevant outcomes including ICU length of stay, ventilator days, and cardiac arrest/extracorporeal membrane oxygenation/mortality. Vasoactive-Inotropic Score may be a useful surrogate outcome in pediatric sepsis.
| S-EPMC5548505 | biostudies-literature
Project description:ImportanceThe Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) uses the Sequential Organ Failure Assessment (SOFA) score to grade organ dysfunction in adult patients with suspected infection. However, the SOFA score is not adjusted for age and therefore not suitable for children.ObjectivesTo adapt and validate a pediatric version of the SOFA score (pSOFA) in critically ill children and to evaluate the Sepsis-3 definitions in patients with confirmed or suspected infection.Design, setting, and participantsThis retrospective observational cohort study included all critically ill children 21 years or younger admitted to a 20-bed, multidisciplinary, tertiary pediatric intensive care unit between January 1, 2009 and August 1, 2016. Data on these children were obtained from an electronic health record database. The pSOFA score was developed by adapting the original SOFA score with age-adjusted cutoffs for the cardiovascular and renal systems and by expanding the respiratory criteria to include noninvasive surrogates of lung injury. Daily pSOFA scores were calculated from admission until day 28 of hospitalization, discharge, or death (whichever came first). Three additional pediatric organ dysfunction scores were calculated for comparison.ExposuresOrgan dysfunction measured by the pSOFA score, and sepsis and septic shock according to the Sepsis-3 definitions.Main outcomes and measuresThe primary outcome was in-hospital mortality. The daily pSOFA scores and additional pediatric organ dysfunction scores were compared. Performance was evaluated using the area under the curve. The pSOFA score was then used to assess the Sepsis-3 definitions in the subgroup of children with confirmed or suspected infection.ResultsIn all, 6303 patients with 8711 encounters met inclusion criteria. Each encounter was treated independently. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. In-hospital mortality was 2.6%. The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an area under the curve of 0.94 (95% CI, 0.92-0.95). The pSOFA score had a similar or better performance than other pediatric organ dysfunction scores. According to the Sepsis-3 definitions, 1231 patients (14.1%) were classified as having sepsis and had a mortality rate of 12.1%, and 347 (4.0%) had septic shock and a mortality rate of 32.3%. Patients with sepsis were more likely to die than patients with confirmed or suspected infection but no sepsis (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.Conclusions and relevanceThe pSOFA score was adapted and validated with age-adjusted variables in critically ill children. Using the pSOFA score, the Sepsis-3 definitions were assessed in children with confirmed or suspected infection. This study is the first assessment, to date, of the Sepsis-3 definitions in critically ill children. Use of these definitions in children is feasible and shows promising results.
| S-EPMC6583375 | biostudies-literature
Project description:Septic shock is induced by an uncontrolled inflammatory immune response to pathogens and the survival rate of patients with pediatric septic shock (PSS) is particularly low, with a mortality rate of 25‑50%. The present study explored the mechanisms of PSS using four microarray datasets (GSE26378, GSE26440, GSE13904 and GSE4607) that were obtained from the Gene Expression Omnibus database. Based on the MetaDE package, the consistently differentially expressed genes (DEGs) in the four datasets were screened. Using the WGCNA package, the disease‑associated modules and genes were identified. Subsequently, the optimal feature genes were further selected using the caret package. Finally, a support vector machine (SVM) classifier based on the optimal feature genes was built using the e1071 package. Initially, there were 2,699 consistent DEGs across the four datasets. From the 10 significantly stable modules across the datasets, four stable modules (including the magenta, purple, turquoise and yellow modules), in which the consistent DEGs were significantly enriched (P<0.05), were further screened. Subsequently, six optimal feature genes (including cysteine rich transmembrane module containing 1, S100 calcium binding protein A9, solute carrier family 2 member 14, stomatin, uridine phosphorylase 1 and utrophin) were selected from the genes in the four stable modules. Additionally, an effective SVM classifier was constructed based on the six optimal genes. The SVM classifier based on the six optimal genes has the potential to be applied for PSS diagnosis. This may improve the accuracy of early PSS diagnosis and suggest possible molecular targets for interventions.
| S-EPMC7003034 | biostudies-literature