Are prognostic tools losing accuracy? Development and performance of a novel age-calibrated severity scoring system for critically ill patients.
Ontology highlight
ABSTRACT:
SUBMITTER: Menezes RC
PROVIDER: S-EPMC7641388 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Are prognostic tools losing accuracy? Development and performance of a novel age-calibrated severity scoring system for critically ill patients.
Menezes Rodrigo C RC Ferreira Isabella B B IBB Carmo Thomas A TA Telles Gabriel P GP Pugas Paula L D PLD Otero Matheus L ML Arriaga Maria B MB Fukutani Kiyoshi F KF Neto Licurgo P LP Agareno Sydney S Filgueiras Filho Nivaldo M NM Akrami Kevan M KM Andrade Bruno B BB
PloS one 20201104 11
<h4>Objective</h4>This study aimed to assess the performance of a commonly used ICU severity score (SAPS3) and determine whether an alternative scoring system may be more accurate across all age strata.<h4>Methods</h4>Retrospective cohort study in a general ICU in Brazil. A secondary analysis was performed with clinical and epidemiological data, present in the first 24 hours of unit admission. Then, a binary logistic regression, followed by cross-validation, was made to develop a novel prognosti ...[more]
Similar Datasets
Project description:Illness severity scores are regularly employed for quality improvement and benchmarking in the intensive care unit, but poor generalization performance, particularly with respect to probability calibration, has limited their use for decision support. These models tend to perform worse in patients at a high risk for mortality. We hypothesized that a sequential modeling approach wherein an initial regression model assigns risk and all patients deemed high risk then have their risk quantified by a second, high-risk-specific, regression model would result in a model with superior calibration across the risk spectrum. We compared this approach to a logistic regression model and a sophisticated machine learning approach, the gradient boosting machine. The sequential approach did not have an effect on the receiver operating characteristic curve or the precision-recall curve but resulted in improved reliability curves. The gradient boosting machine achieved a small improvement in discrimination performance and was similarly calibrated to the sequential models.
| S-EPMC6695410 | biostudies-literature
Project description:Among critical patients, few studies have evaluated the discrimination of current illness scoring systems in predicting outcomes after continuous renal replacement therapy (CRRT) initiation. Patients receiving CRRT in the ICU between 2005 and 2018 from the Chang Gung Research Database were extracted. All the components of the Acute Physiology Assessment and Chronic Health Evaluation (APACHE) III, Sequential Organ Failure Assessment (SOFA), qSOFA, and MOSAIC scoring systems on days 1, 3, and 7 of CRRT were recorded. Patients older than 80 years were identified and analyzed separately. We identified 3370 adult patients for analysis. The discrimination ability of the scoring systems was acceptable at day 7 after CRRT initiation, including SOFA (area under the receiver operating characteristic curve, 74.1% (95% confidence interval, 71.7-76.5%)), APACHEIII (74.7% (72.3-77.1%)), and MOSAIC (71.3% (68.8%-73.9%)). These systems were not ideal on days 1 and 3, and that of qSOFA was poor at any time point. The discrimination performance was slightly better among patients ≥80 years. APACHE III, MOSAIC, and SOFA can be intensivists and families' reference to make their decision of withdrawing or withholding CRRT after a short period of treatment, especially in adults ≥80 years old.
| S-EPMC8509572 | biostudies-literature
Project description:Many injury severity scoring tools have been developed over the past few decades. These tools include the Injury Severity Score (ISS), New ISS (NISS), Trauma and Injury Severity Score (TRISS) and International Classification of Diseases (ICD)-based Injury Severity Score (ICISS). Although many studies have endeavored to determine the ability of these tools to predict the mortality of injured patients, their results have been inconsistent. We conducted a systematic review to summarize the predictive performances of these tools and explore the heterogeneity among studies. We defined a relevant article as any research article that reported the area under the Receiver Operating Characteristic curve as a measure of predictive performance. We conducted an online search using MEDLINE and Embase. We evaluated the quality of each relevant article using a quality assessment questionnaire consisting of 10 questions. The total number of positive answers was reported as the quality score of the study. Meta-analysis was not performed due to the heterogeneity among studies. We identified 64 relevant articles with 157 AUROCs of the tools. The median number of positive answers to the questionnaire was 5, ranging from 2 to 8. Less than half of the relevant studies reported the version of the Abbreviated Injury Scale (AIS) and/or ICD (37.5%). The heterogeneity among the studies could be observed in a broad distribution of crude mortality rates of study data, ranging from 1% to 38%. The NISS was mostly reported to perform better than the ISS when predicting the mortality of blunt trauma patients. The relative performance of the ICSS against the AIS-based tools was inconclusive because of the scarcity of studies. The performance of the ICISS appeared to be unstable because the performance could be altered by the type of formula and survival risk ratios used. In conclusion, high-quality studies were limited. The NISS might perform better in the mortality prediction of blunt injuries than the ISS. Additional studies are required to standardize the derivation of the ICISS and determine the relative performance of the ICISS against the AIS-based tools.
| S-EPMC3511252 | biostudies-literature
Project description:Introduction: Pain impacts brain development for neonates, causing deleterious neurodevelopmental outcomes. Prescription opioids for analgesia or sedation are common; however, prolonged opioid exposure in neonates is associated with neurodevelopmental impairment. Balancing the impact of inadequate pain control against prolonged opioid exposure in neonates is a clinical paradox. Therefore, we sought to decrease the average days of opioids used for analgesia or sedation in critically ill neonates at a level IV Neonatal Intensive Care Unit by 10% within 1 year. Methods: A multidisciplinary quality improvement team used the model for improvement, beginning with a Pareto analysis, and identified a lack of consistent approach to weaning opioids as a primary driver for prolonged exposure. The team utilized 2 main interventions: (1) a standardized withdrawal assessment tool-1 and (2) a risk-stratified opioid weaning guideline. Results: We demonstrated a reduction in mean opioid duration from 34.3 to 14.1 days, an increase in nursing withdrawal assessment tool-1 documentation from 20% to 90%, and an increase in the documented rationale for daily opioid dose in provider notes from 20% to 70%. Benzodiazepine use did not change. Conclusion: Standardized withdrawal assessments combined with risk-stratified weaning guidelines can decrease opioid use in critically ill neonates.
| S-EPMC9197367 | biostudies-literature
Project description:Objective:Electrographic seizures in critically ill patients are often equivocal. In this study, we sought to determine the diagnostic accuracy of electrographic seizure annotation in adult intensive care units (ICUs) and to identify affecting factors. Methods:To investigate diagnostic accuracy, interreader agreement (IRA) measures were derived from 5,769 unequivocal and 6,263 equivocal seizure annotations by five experienced electroencephalogram (EEG) readers after reviewing 74 days of EEGs from 50 adult ICU patients. Factors including seizure equivocality (unequivocal vs. equivocal) and laterality (generalized, partial, or bilaterally independent), cyclicity (cyclic vs. noncyclic), persistency (occurrence of status epilepticus), and patient consciousness level (coma vs. noncoma) were further investigated for their influence on IRA measures. Results:On average, 70% of seizures marked by a reference reader overlapped, at least in part, with those marked by a test reader (any-overlap sensitivity, AO-Sn). Agreed seizure duration between reader pairs (overlap-integral sensitivity, OI-Sn) was 62%, while agreed nonseizure duration (overlap-integral specificity, OI-Sp) was 99%. A test reader would annotate one additional seizure not overlapping with a reference reader's annotation in every 11.7 h of EEG, that is, the false-positive rate (FPR) was 0.0854/h. Classifying seizure patterns into unequivocal and equivocal improved specificity and FPR (unequivocal patterns) but compromised sensitivity only for equivocal patterns. Sensitivity of all and unequivocal annotations was higher for patients with status epilepticus. Specificity was higher for partial than for bilaterally independent unequivocal seizure patterns, and lower for cyclic all seizure patterns. Significance:Diagnosing electrographic seizures in critically ill adults is highly specific and moderately sensitive. Improved criteria for diagnosing electrographic seizures in the ICU are needed.
| S-EPMC5939393 | biostudies-literature
Project description:ObjectivesTo determine clinician accuracy in the identification and prediction of multiple organ dysfunction syndrome.DesignProspective cohort study.SettingUniversity of Michigan's C.S. Mott Children's Hospital PICU.PatientsPatients admitted to the PICU with an anticipated PICU length of stay greater than 48 hours.InterventionsNone.Measurements and main resultsFor each patient, the clinical team (attending, fellow, resident/nurse practitioner) was surveyed regarding existing and anticipated organ dysfunction. The primary outcomes were clinicians' accuracy at identifying multiple organ dysfunction syndrome and predicting new or progressive multiple organ dysfunction syndrome, compared to the objective assessment of multiple organ dysfunction syndrome using Proulx criteria. We also measured sensitivity, specificity, negative and positive predictive values, and negative and positive likelihood ratios of clinician assessments. We tested for differences in accuracy by clinician type using chi-square tests. Clinicians rated their confidence in prediction on a 5-point Likert scale. There were 476 eligible PICU admissions, for whom 1,218 surveys were completed. Multiple organ dysfunction syndrome was present in 89 patients (18.7%) at enrollment, and new or progressive multiple organ dysfunction syndrome occurred in 39 (8.2%). Clinicians correctly identified multiple organ dysfunction syndrome with 79.9% accuracy and predicted additional organ dysfunction with 82.6% accuracy. However, the positive and negative likelihood ratios for new or progressive multiple organ dysfunction syndrome prediction were 3.0 and 0.7, respectively, indicating a weak relationship between the clinician prediction and development of new or progressive multiple organ dysfunction syndrome. The positive predictive value of new or progressive multiple organ dysfunction syndrome prediction was just 22.1%. We found no differences in accuracy by clinician type for either identification of multiple organ dysfunction syndrome (80.2% vs 78.2% vs 81.0%; p = 0.57) or prediction of new or progressive multiple organ dysfunction syndrome (84.8% vs 82.8% vs 80.3%; p = 0.26) for attendings, fellows, and residents/nurse practitioners, respectively. There was a weak correlation between the confidence and accuracy of prediction (pairwise correlation coefficient, 0.26; p < 0.001).ConclusionsPICU clinicians correctly identified multiple organ dysfunction syndrome and predicted new or progressive multiple organ dysfunction syndrome with 80% accuracy. However, only 8% of patients developed new or progressive multiple organ dysfunction syndrome, so accuracy was largely due to true negative predictions. The positive predictive value for new or progressive multiple organ dysfunction syndrome prediction was just 22%. Accuracy did not differ by clinician type, but was correlated with self-rated confidence and was higher for negative predictions.
| S-EPMC7959260 | biostudies-literature
Project description:ObjectivesAmylase concentration in respiratory secretions was reported to be a potentially useful marker for aspiration and pneumonia. The aim of this study was to determine accuracy of α-amylase in diagnosing microaspiration in critically ill patients.MethodsRetrospective analysis of prospectively collected data collected in a medical ICU. All patients requiring mechanical ventilation for at least 48 h, and included in a previous randomized controlled trial were eligible for this study, provided that at least one tracheal aspirate was available for α-amylase measurement. As part of the initial trial, pepsin was quantitatively measured in all tracheal aspirates during a 48-h period. All tracheal aspirates were frozen, allowing subsequent measurement of α-amylase for the purpose of the current study. Microaspiration was defined as the presence of at least one positive tracheal aspirate for pepsin (>200 ng.mL-1). Abundant microaspiration was defined as the presence of pepsin at significant level in >74% of tracheal aspirates.ResultsAmylase was measured in 1055 tracheal aspirates, collected from 109 patients. Using mean α-amylase level per patient, accuracy of α-amylase in diagnosing microaspiration was moderate (area under the receiver operator curve 0.72±0.05 [95%CI 0.61-0.83], for an α-amylase value of 1685 UI.L-1). However, when α-amylase levels, coming from all samples, were taken into account, area under the receiver operator curve was 0.56±0.05 [0.53-0.60]. Mean α-amylase level, and percentage of tracheal aspirates positive for α-amylase were significantly higher in patients with microaspiration, and in patients with abundant microaspiration compared with those with no microaspiration; and similar in patients with microaspiration compared with those with abundant microaspiration. α-amylase and pepsin were significantly correlated (r2 = 0.305, p = 0.001).ConclusionAccuracy of mean α-amylase in diagnosing microaspiration is moderate. Further, when all α-amylase levels were taken into account, α-amylase was inaccurate in diagnosing microaspiration, compared with pepsin.
| S-EPMC3946401 | biostudies-literature
Project description:ObjectivesDNA methylation can be used to determine an individual's biological age, as opposed to chronological age, an indicator of underlying health status. This study aimed to assess epigenetic age in critically ill patients with and without sepsis to determine if higher epigenetic age is associated with admission diagnosis or mortality.DesignSecondary analysis of whole blood DNA methylation data generated from a nested case-control study of critically ill septic and nonseptic patients.SettingFour tertiary care hospitals in Canada.InterventionsNone.PatientsCritically ill patients with and without sepsis.Measurements and main resultsEpigenetic age was derived from DNA methylation data using the Hannum and PhenoAge algorithms and deviation from the patient's chronological age in years was determined. Of the 66 patients with sepsis, 34 were male (51.5%), the mean age was 65.03 years and 25 patients (37.8%) died before discharge. Of the 68 nonseptic patients, 47 were male (69.1%), the mean age was 64.92 years and 25 (36.7%) died before discharge. Epigenetic age calculated using the PhenoAge algorithm showed a significant age acceleration of 4.97 years in septic patients (p = 0.045), but no significant acceleration in nonseptic patients. Epigenetic age calculated using the Hannum algorithm showed no significant acceleration in the septic or nonseptic patients. Similarly, in the combined septic and nonseptic cohorts, nonsurvivors showed an epigenetic age acceleration of 7.62 years (p = 0.004) using the PhenoAge algorithm while survivors showed no significant age acceleration. Survivor status was not associated with age acceleration using the Hannum algorithm.ConclusionsIn critically ill patients, epigenetic age acceleration, as calculated by the PhenoAge algorithm, was associated with sepsis diagnosis and mortality.
| S-EPMC10857665 | biostudies-literature
Project description:Background: Systemic inflammation is a whole body reaction that can have an infection-positive (i.e. sepsis) or infection-negative origin. It is important to distinguish between septic and non-septic presentations early and reliably, because this has significant therapeutic implications for critically ill patients. We hypothesized that a molecular classifier based on a small number of RNAs expressed in peripheral blood could be discovered that would: 1) determine which patients with systemic inflammation had sepsis; 2) be robust across independent patient cohorts; 3) be insensitive to disease severity; and 4) provide diagnostic utility. The overall goal of this study was to identify and validate such a molecular classifier. Methods and Findings: We conducted an observational, non-interventional study of adult patients recruited from tertiary intensive care units (ICU). Biomarker discovery was conducted with an Australian cohort (n = 105) consisting of sepsis patients and post -surgical patients with infection-negative systemic inflammation. Using this cohort, a four-gene classifier consisting of a combination of CEACAM4, LAMP1, PLA2G7 and PLAC8 RNA biomarkers was identified. This classifier, designated SeptiCyte® Lab, was externally validated using RT-qPCR and receiver operating characteristic (ROC) curve analysis in five cohorts (n = 345) from the Netherlands. Cohort 1 (n=59) consisted of unambiguous septic cases and infection-negative systemic inflammation controls; SeptiCyte® Lab gave an area under curve (AUC) of 0.96 (95% CI: 0.91-1.00). ROC analysis of a more heterogeneous group of patients (Cohorts 2-5; 249 patients after excluding 37 patients with infection likelihood possible) gave an AUC of 0.89 (95% CI: 0.85-0.93). Disease severity, as measured by Sequential Organ Failure Assessment (SOFA) score or the Acute Physiology and Chronic Health Evaluation (APACHE) IV score, was not a significant confounding variable. The diagnostic utility o f SeptiCyte® Lab was evaluated by comparison to various clinical and laboratory parameters that would be available to a clinician within 24 hours of ICU admission. SeptiCyte® Lab was significantly better at differentiating sepsis from infection-negative systemic inflammation than all tested parameters, both singly and in various logistic combinations. SeptiCyte® Lab more than halved the diagnostic error rate compared to PCT in all tested cohorts or cohort combinations. Conclusions: SeptiCyte® Lab is a rapid molecular assay that may be clinically useful in the management of ICU patients with systemic inflammation. SIRS and Sepsis ICU patients, admission samples Retrospective, mutli-site sutdy using retrospective physician adjudication as a comparator
2015-10-24 | E-GEOD-74224 | biostudies-arrayexpress
Project description:Background and aimsRecently, glucose variability (GV) has been reported as an independent risk factor for mortality in non-diabetic critically ill patients. However, GV is not incorporated in any severity scoring system for critically ill patients currently. The aim of this study was to establish and validate a modified Simplified Acute Physiology Score II scoring system (SAPS II), integrated with GV parameters and named GV-SAPS II, specifically for non-diabetic critically ill patients to predict short-term and long-term mortality.MethodsTraining and validation cohorts were exacted from the Multiparameter Intelligent Monitoring in Intensive Care database III version 1.3 (MIMIC-III v1.3). The GV-SAPS II score was constructed by Cox proportional hazard regression analysis and compared with the original SAPS II, Sepsis-related Organ Failure Assessment Score (SOFA) and Elixhauser scoring systems using area under the curve of the receiver operator characteristic (auROC) curve.Results4,895 and 5,048 eligible individuals were included in the training and validation cohorts, respectively. The GV-SAPS II score was established with four independent risk factors, including hyperglycemia, hypoglycemia, standard deviation of blood glucose levels (GluSD), and SAPS II score. In the validation cohort, the auROC values of the new scoring system were 0.824 (95% CI: 0.813-0.834, P< 0.001) and 0.738 (95% CI: 0.725-0.750, P< 0.001), respectively for 30 days and 9 months, which were significantly higher than other models used in our study (all P < 0.001). Moreover, Kaplan-Meier plots demonstrated significantly worse outcomes in higher GV-SAPS II score groups both for 30-day and 9-month mortality endpoints (all P< 0.001).ConclusionsWe established and validated a modified prognostic scoring system that integrated glucose variability for non-diabetic critically ill patients, named GV-SAPS II. It demonstrated a superior prognostic capability and may be an optimal scoring system for prognostic evaluation in this patient group.
| S-EPMC5100948 | biostudies-literature