Development and external validation of a COVID-19 mortality risk prediction algorithm: a multicentre retrospective cohort study.
Ontology highlight
ABSTRACT: Objective
This study aimed to develop and externally validate a COVID-19 mortality risk prediction algorithm.Design
Retrospective cohort study.Setting
Five designated tertiary hospitals for COVID-19 in Hubei province, China.Participants
We routinely collected medical data of 1364 confirmed adult patients with COVID-19 between 8 January and 19 March 2020. Among them, 1088 patients from two designated hospitals in Wuhan were used to develop the prognostic model, and 276 patients from three hospitals outside Wuhan were used for external validation. All patients were followed up for a maximal of 60 days after the diagnosis of COVID-19.Methods
The model discrimination was assessed by the area under the receiver operating characteristic curve (AUC) and Somers' D test, and calibration was examined by the calibration plot. Decision curve analysis was conducted.Main outcome measures
The primary outcome was all-cause mortality within 60 days after the diagnosis of COVID-19.Results
The full model included seven predictors of age, respiratory failure, white cell count, lymphocytes, platelets, D-dimer and lactate dehydrogenase. The simple model contained five indicators of age, respiratory failure, coronary heart disease, renal failure and heart failure. After cross-validation, the AUC statistics based on derivation cohort were 0.96 (95% CI, 0.96 to 0.97) for the full model and 0.92 (95% CI, 0.89 to 0.95) for the simple model. The AUC statistics based on the external validation cohort were 0.97 (95% CI, 0.96 to 0.98) for the full model and 0.88 (95% CI, 0.80 to 0.96) for the simple model. Good calibration accuracy of these two models was found in the derivation and validation cohort.Conclusion
The prediction models showed good model performance in identifying patients with COVID-19 with a high risk of death in 60 days. It may be useful for acute risk classification.Web calculator
We provided a freely accessible web calculator (https://www.whuyijia.com/).
SUBMITTER: Mei J
PROVIDER: S-EPMC7768618 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundPublic policy measures and clinical risk assessments relevant to COVID-19 need to be aided by risk prediction models that are rigorously developed and validated. We aimed to externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.MethodsWe did a population-based cohort study using the UK Office for National Statistics Public Health Linked Data Asset, a cohort of individuals aged 19-100 years, based on the 2011 census and linked to Hospital Episode Statistics, the General Practice Extraction Service data for pandemic planning and research, and radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two periods were used: (1) Jan 24 to April 30, 2020, and (2) May 1 to July 28, 2020. We assessed the performance of the QCovid algorithms using measures of discrimination and calibration. Using predicted 90-day risk of COVID-19 death, we calculated r2 values, Brier scores, and measures of discrimination and calibration with corresponding 95% CIs over the two time periods.FindingsWe included 34 897 648 adults aged 19-100 years resident in England. 26 985 (0·08%) COVID-19 deaths occurred during the first period and 13 177 (0·04%) during the second. The algorithms had good discrimination and calibration in both periods. In the first period, they explained 77·1% (95% CI 76·9-77·4) of the variation in time to death in men and 76·3% (76·0-76·6) in women. The D statistic was 3·761 (3·732-3·789) for men and 3·671 (3·640-3·702) for women and Harrell's C was 0·935 (0·933-0·937) for men and 0·945 (0·943-0·947) for women. Similar results were obtained for the second time period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first period was 65·94% for men and 71·67% for women.InterpretationThe QCovid population-based risk algorithm performed well, showing high levels of discrimination for COVID-19 deaths in men and women for both time periods. QCovid has the potential to be dynamically updated as the pandemic evolves and, therefore, has potential use in guiding national policy.FundingUK National Institute for Health Research.
| S-EPMC8148652 | biostudies-literature
Project description:OBJECTIVES:Being able to predict which patients with COVID-19 are going to deteriorate is important to help identify patients for clinical and research practice. Clinical prediction models play a critical role in this process, but current models are of limited value because they are typically restricted to baseline predictors and do not always use contemporary statistical methods. We sought to explore the benefits of incorporating dynamic changes in routinely measured biomarkers, non-linear effects and applying 'state-of-the-art' statistical methods in the development of a prognostic model to predict death in hospitalised patients with COVID-19. DESIGN:The data were analysed from admissions with COVID-19 to three hospital sites. Exploratory data analysis included a graphical approach to partial correlations. Dynamic biomarkers were considered up to 5 days following admission rather than depending solely on baseline or single time-point data. Marked departures from linear effects of covariates were identified by employing smoothing splines within a generalised additive modelling framework. SETTING:3 secondary and tertiary level centres in Greater Manchester, the UK. PARTICIPANTS:392 hospitalised patients with a diagnosis of COVID-19. RESULTS:392 patients with a COVID-19 diagnosis were identified. Area under the receiver operating characteristic curve increased from 0.73 using admission data alone to 0.75 when also considering results of baseline blood samples and to 0.83 when considering dynamic values of routinely collected markers. There was clear non-linearity in the association of age with patient outcome. CONCLUSIONS:This study shows that clinical prediction models to predict death in hospitalised patients with COVID-19 can be improved by taking into account both non-linear effects in covariates such as age and dynamic changes in values of biomarkers.
| S-EPMC7513423 | biostudies-literature
Project description:ObjectivesPredicting diagnosis and prognosis of traumatic brain injury (TBI) at the prehospital stage is challenging; however, using comprehensive prehospital information and machine learning may improve the performance of the predictive model. We developed and tested predictive models for TBI that use machine learning algorithms using information that can be obtained in the prehospital stage.DesignThis was a multicentre retrospective study.Setting and participantsThis study was conducted at three tertiary academic emergency departments (EDs) located in an urban area of South Korea. The data from adult patients with severe trauma who were assessed by emergency medical service providers and transported to three participating hospitals between 2014 to 2018 were analysed.ResultsWe developed and tested five machine learning algorithms-logistic regression analyses, extreme gradient boosting, support vector machine, random forest and elastic net (EN)-to predict TBI, TBI with intracranial haemorrhage or injury (TBI-I), TBI with ED or admission result of admission or transferred (TBI with non-discharge (TBI-ND)) and TBI with ED or admission result of death (TBI-D). A total of 1169 patients were included in the final analysis, and the proportions of TBI, TBI-I, TBI-ND and TBI-D were 24.0%, 21.5%, 21.3% and 3.7%, respectively. The EN model yielded an area under receiver-operator curve of 0.799 for TBI, 0.844 for TBI-I, 0.811 for TBI-ND and 0.871 for TBI-D. The EN model also yielded the highest specificity and significant reclassification improvement. Variables related to loss of consciousness, Glasgow Coma Scale and light reflex were the three most important variables to predict all outcomes.ConclusionOur results inform the diagnosis and prognosis of TBI. Machine learning models resulted in significant performance improvement over that with logistic regression analyses, and the best performing model was EN.
| S-EPMC8756263 | biostudies-literature
Project description:Two neonatal mortality prediction models, the Neonatal Essential Treatment Score (NETS) which uses treatments prescribed at admission and the Score for Essential Neonatal Symptoms and Signs (SENSS) which uses basic clinical signs, were derived in high-mortality, low-resource settings to utilise data more likely to be available in these settings. In this study, we evaluate the predictive accuracy of two neonatal prediction models for all-cause in-hospital mortality. We used retrospectively collected routine clinical data recorded by duty clinicians at admission from 16 Kenyan hospitals used to externally validate and update the SENSS and NETS models that were initially developed from the data from the largest Kenyan maternity hospital to predict in-hospital mortality. Model performance was evaluated by assessing discrimination and calibration. Discrimination, the ability of the model to differentiate between those with and without the outcome, was measured using the c-statistic. Calibration, the agreement between predictions from the model and what was observed, was measured using the calibration intercept and slope (with values of 0 and 1 denoting perfect calibration). At initial external validation, the estimated mortality risks from the original SENSS and NETS models were markedly overestimated with calibration intercepts of - 0.703 (95% CI - 0.738 to - 0.669) and - 1.109 (95% CI - 1.148 to - 1.069) and too extreme with calibration slopes of 0.565 (95% CI 0.552 to 0.577) and 0.466 (95% CI 0.451 to 0.480), respectively. After model updating, the calibration of the model improved. The updated SENSS and NETS models had calibration intercepts of 0.311 (95% CI 0.282 to 0.350) and 0.032 (95% CI - 0.002 to 0.066) and calibration slopes of 1.029 (95% CI 1.006 to 1.051) and 0.799 (95% CI 0.774 to 0.823), respectively, while showing good discrimination with c-statistics of 0.834 (95% CI 0.829 to 0.839) and 0.775 (95% CI 0.768 to 0.782), respectively. The overall calibration performance of the updated SENSS and NETS models was better than any existing neonatal in-hospital mortality prediction models externally validated for settings comparable to Kenya. Few prediction models undergo rigorous external validation. We show how external validation using data from multiple locations enables model updating and improving their performance and potential value. The improved models indicate it is possible to predict in-hospital mortality using either treatments or signs and symptoms derived from routine neonatal data from low-resource hospital settings also making possible their use for case-mix adjustment when contrasting similar hospital settings.
| S-EPMC9347100 | biostudies-literature
Project description:ObjectivesWe aimed to examine the validity of the quick Sequential Organ Failure Assessment (qSOFA) score for mortality and bacteraemia risk assessment in Japanese haemodialysis patients.DesignThis is a retrospective multicentre cohort study.SettingThe six participating hospitals are tertiary-care institutions that receive patients on an emergency basis and provide primary, secondary and tertiary care. The other participating hospital is a secondary-care institution that receives patients on an emergency basis and provides both primary and secondary care.ParticipantsThis study included haemodialysis outpatients admitted for bacteraemia suspicion, who had blood drawn for cultures within 48 hours of their initial admission.Primary and secondary outcome measuresThe primary outcome measure was overall in-hospital mortality. Secondary outcomes included 28-day in-hospital mortality and the incidence of bacteraemia diagnosed based on blood culture findings. The discrimination, calibration and test performance of the qSOFA score were assessed. Missing data were handled using multiple imputation.ResultsAmong the 507 haemodialysis patients admitted with bacteraemia suspicion between August 2011 and July 2013, the overall in-hospital mortality was 14.6% (74/507), the 28-day in-hospital mortality was 11.1% (56/507) and the incidence of bacteraemia, defined as a positive blood culture, was 13.4% (68/507). For predicting in-hospital mortality among haemodialysis patients, the area under the receiver operating characteristic curve was 0.61 (95% CI 0.56-0.67) for a qSOFA score ≥2. The Hosmer-Lemeshow χ2 statistics for the qSOFA score as a predictor of overall and 28-day in-hospital mortality were 5.72 (p=0.02) and 7.40 (p<0.01), respectively.ConclusionOn external validation, the qSOFA score exhibited low diagnostic accuracy and miscalibration for in-hospital mortality and bacteraemia among haemodialysis patients.
| S-EPMC6629386 | biostudies-literature
Project description:ObjectiveTo develop and externally validate the STRAtifying Treatments In the multi-morbid Frail elderlY (STRATIFY)-Falls clinical prediction model to identify the risk of hospital admission or death from a fall in patients with an indication for antihypertensive treatment.DesignRetrospective cohort study.SettingPrimary care data from electronic health records contained within the UK Clinical Practice Research Datalink (CPRD).ParticipantsPatients aged 40 years or older with at least one blood pressure measurement between 130 mm Hg and 179 mm Hg.Main outcome measureFirst serious fall, defined as hospital admission or death with a primary diagnosis of a fall within 10 years of the index date (12 months after cohort entry). Model development was conducted using a Fine-Gray approach in data from CPRD GOLD, accounting for the competing risk of death from other causes, with subsequent recalibration at one, five, and 10 years using pseudo values. External validation was conducted using data from CPRD Aurum, with performance assessed through calibration curves and the observed to expected ratio, C statistic, and D statistic, pooled across general practices, and clinical utility using decision curve analysis at thresholds around 10%.ResultsAnalysis included 1 772 600 patients (experiencing 62 691 serious falls) from CPRD GOLD used in model development, and 3 805 366 (experiencing 206 956 serious falls) from CPRD Aurum in the external validation. The final model consisted of 24 predictors, including age, sex, ethnicity, alcohol consumption, living in an area of high social deprivation, a history of falls, multiple sclerosis, and prescriptions of antihypertensives, antidepressants, hypnotics, and anxiolytics. Upon external validation, the recalibrated model showed good discrimination, with pooled C statistics of 0.833 (95% confidence interval 0.831 to 0.835) and 0.843 (0.841 to 0.844) at five and 10 years, respectively. Original model calibration was poor on visual inspection and although this was improved with recalibration, under-prediction of risk remained (observed to expected ratio at 10 years 1.839, 95% confidence interval 1.811 to 1.865). Nevertheless, decision curve analysis suggests potential clinical utility, with net benefit larger than other strategies.ConclusionsThis prediction model uses commonly recorded clinical characteristics and distinguishes well between patients at high and low risk of falls in the next 1-10 years. Although miscalibration was evident on external validation, the model still had potential clinical utility around risk thresholds of 10% and so could be useful in routine clinical practice to help identify those at high risk of falls who might benefit from closer monitoring or early intervention to prevent future falls. Further studies are needed to explore the appropriate thresholds that maximise the model's clinical utility and cost effectiveness.
| S-EPMC9641577 | biostudies-literature
Project description:IntroductionDue to the increased use of CT and MRI, the prevalence of incidental findings on brain scans is increasing. Meningioma, the most common primary brain tumour, is a frequently encountered incidental finding, with an estimated prevalence of 3/1000. The management of incidental meningioma varies widely with active clinical-radiological monitoring being the most accepted method by clinicians. Duration of monitoring and time intervals for assessment, however, are not well defined. To this end, we have recently developed a statistical model of progression risk based on single-centre retrospective data. The model Incidental Meningioma: Prognostic Analysis Using Patient Comorbidity and MRI Tests (IMPACT) employs baseline clinical and imaging features to categorise the patient with an incidental meningioma into one of three risk groups: low, medium and high risk with a proposed active monitoring strategy based on the risk and temporal trajectory of progression, accounting for actuarial life expectancy. The primary aim of this study is to assess the external validity of this model.Methods and analysisIMPACT is a retrospective multicentre study which will aim to include 1500 patients with an incidental intracranial meningioma, powered to detect a 10% progression risk. Adult patients ≥16 years diagnosed with an incidental meningioma between 1 January 2009 and 31 December 2010 will be included. Clinical and radiological data will be collected longitudinally until the patient reaches one of the study endpoints: intervention (surgery, stereotactic radiosurgery or fractionated radiotherapy), mortality or last date of follow-up. Data will be uploaded to an online Research Electronic Data Capture database with no unique identifiers. External validity of IMPACT will be tested using established statistical methods.Ethics and disseminationLocal institutional approval at each participating centre will be required. Results of the study will be reported through peer-reviewed articles and conferences and disseminated to participating centres, patients and the public using social media.
| S-EPMC8768908 | biostudies-literature
Project description:BackgroundDistinguishing tuberculous pleural effusion (TPE) from non-tuberculosis (TB) benign pleural effusion (BPE) remains to be a challenge in clinical practice. The aim of the present study was to develop and validate a novel nomogram for diagnosing TPE.MethodsIn this retrospective analysis, a total of 909 consecutive patients with TPE and non-TB BPE from Ningbo First Hospital were divided into the training set and the internal validation set at a ratio of 7:3, respectively. The clinical and laboratory features were collected and analyzed by logistic regression analysis. A diagnostic model incorporating selected variables was developed and was externally validated in a cohort of 110 patients from another hospital.ResultsSix variables including age, effusion lymphocyte, effusion adenosine deaminase (ADA), effusion lactatedehy drogenase (LDH), effusion LDH/effusion ADA, and serum white blood cell (WBC) were identified as valuable parameters used for developing a nomogram. The nomogram showed a good diagnostic performance in the training set. A novel scoring system was then established based on the nomogram to distinguish TPE from non-TB BPE. The scoring system showed good diagnostic performance in the training set [area under the curve (AUC) (95% confidence interval (CI)), 0.937 (0.917-0.957); sensitivity, 89.0%, and specificity, 89.5%], the internal validation set [AUC (95%CI), 0.934 (0.902-0.966); sensitivity, 88.7%, and specificity, 90.3%], and the external validation set [(AUC (95%CI), 0.941 (0.891-0.991); sensitivity, 93.6%, and specificity, 87.5%)], respectively.ConclusionsThe study developed and validated a novel scoring system based on a nomogram originated from six clinical parameters. The novel scoring system showed a good diagnostic performance in distinguishing TPE from non-TB BPE and can be conveniently used in clinical settings.
| S-EPMC9145463 | biostudies-literature
Project description:BackgroundIdentifying patients at risk for mortality from COVID-19 is crucial to triage, clinical decision-making, and the allocation of scarce hospital resources. The 4C Mortality Score effectively predicts COVID-19 mortality, but it has not been validated in a United States (U.S.) population. The purpose of this study is to determine whether the 4C Mortality Score accurately predicts COVID-19 mortality in an urban U.S. adult inpatient population.MethodsThis retrospective cohort study included adult patients admitted to a single-center, tertiary care hospital (Philadelphia, PA) with a positive SARS-CoV-2 PCR from 3/01/2020 to 6/06/2020. Variables were extracted through a combination of automated export and manual chart review. The outcome of interest was mortality during hospital admission or within 30 days of discharge.ResultsThis study included 426 patients; mean age was 64.4 years, 43.4% were female, and 54.5% self-identified as Black or African American. All-cause mortality was observed in 71 patients (16.7%). The area under the receiver operator characteristic curve of the 4C Mortality Score was 0.85 (95% confidence interval, 0.79-0.89).ConclusionsClinicians may use the 4C Mortality Score in an urban, majority Black, U.S. inpatient population. The derivation and validation cohorts were treated in the pre-vaccine era so the 4C Score may over-predict mortality in current patient populations. With stubbornly high inpatient mortality rates, however, the 4C Score remains one of the best tools available to date to inform thoughtful triage and treatment allocation.
| S-EPMC9054702 | biostudies-literature
Project description:ImportanceEstimating mortality risk in older adults with dementia is important for guiding decisions such as cancer screening, treatment of new and chronic medical conditions, and advance care planning.ObjectiveTo develop and externally validate a mortality prediction model in community-dwelling older adults with dementia.Design, setting, and participantsThis cohort study included community-dwelling participants (aged ≥65 years) in the Health and Retirement Study (HRS) from 1998 to 2016 (derivation cohort) and National Health and Aging Trends Study (NHATS) from 2011 to 2019 (validation cohort).ExposuresCandidate predictors included demographics, behavioral/health factors, functional measures (eg, activities of daily living [ADL] and instrumental activities of daily living [IADL]), and chronic conditions.Main outcomes and measuresThe primary outcome was time to all-cause death. We used Cox proportional hazards regression with backward selection and multiple imputation for model development. Model performance was assessed by discrimination (integrated area under the receiver operating characteristic curve [iAUC]) and calibration (plots of predicted and observed mortality).ResultsOf 4267 participants with probable dementia in HRS, the mean (SD) age was 82.2 (7.6) years, 2930 (survey-weighted 69.4%) were female, and 785 (survey-weighted 12.1%) identified as Black. Median (IQR) follow-up time was 3.9 (2.0-6.8) years, and 3466 (81.2%) participants died by end of follow-up. The final model included age, sex, body mass index, smoking status, ADL dependency count, IADL difficulty count, difficulty walking several blocks, participation in vigorous physical activity, and chronic conditions (cancer, heart disease, diabetes, lung disease). The optimism-corrected iAUC after bootstrap internal validation was 0.76 (95% CI, 0.75-0.76) with time-specific AUC of 0.73 (95% CI, 0.70-0.75) at 1 year, 0.75 (95% CI, 0.73-0.77) at 5 years, and 0.84 (95% CI, 0.82-0.85) at 10 years. On external validation in NHATS (n = 2404), AUC was 0.73 (95% CI, 0.70-0.76) at 1 year and 0.74 (95% CI, 0.71-0.76) at 5 years. Calibration plots suggested good calibration across the range of predicted risk from 1 to 10 years.Conclusions and relevanceWe developed and externally validated a mortality prediction model in community-dwelling older adults with dementia that showed good discrimination and calibration. The mortality risk estimates may help guide discussions regarding treatment decisions and advance care planning.
| S-EPMC9513707 | biostudies-literature