Predictive performance of international COVID-19 mortality forecasting models.
Ontology highlight
ABSTRACT:
SUBMITTER: Friedman J
PROVIDER: S-EPMC7685335 | biostudies-literature | 2020 Nov
REPOSITORIES: biostudies-literature
Publications
Predictive performance of international COVID-19 mortality forecasting models.
Friedman Joseph J Liu Patrick P Troeger Christopher E CE Carter Austin A Reiner Robert C RC Barber Ryan M RM Collins James J Lim Stephen S SS Pigott David M DM Vos Theo T Hay Simon I SI Murray Christopher J L CJL Gakidou Emmanuela E
medRxiv : the preprint server for health sciences 20201119
Forecasts and alternative scenarios of COVID-19 mortality have been critical inputs into a range of policies and decision-makers need information about predictive performance. We identified n=386 public COVID-19 forecasting models and included n=8 that were global in scope and provided public, date-versioned forecasts. For each, we examined the median absolute percent error (MAPE) compared to subsequently observed mortality trends, stratified by weeks of extrapolation, world region, and month of ...[more]
Similar Datasets
Project description:BackgroundPrevious studies have suggested associations between trends of web searches and COVID-19 traditional metrics. It remains unclear whether models incorporating trends of digital searches lead to better predictions.ObjectiveThe aim of this study is to investigate the relationship between Google Trends searches of symptoms associated with COVID-19 and confirmed COVID-19 cases and deaths. We aim to develop predictive models to forecast the COVID-19 epidemic based on a combination of Google Trends searches of symptoms and conventional COVID-19 metrics.MethodsAn open-access web application was developed to evaluate Google Trends and traditional COVID-19 metrics via an interactive framework based on principal component analysis (PCA) and time series modeling. The application facilitates the analysis of symptom search behavior associated with COVID-19 disease in 188 countries. In this study, we selected the data of nine countries as case studies to represent all continents. PCA was used to perform data dimensionality reduction, and three different time series models (error, trend, seasonality; autoregressive integrated moving average; and feed-forward neural network autoregression) were used to predict COVID-19 metrics in the upcoming 14 days. The models were compared in terms of prediction ability using the root mean square error (RMSE) of the first principal component (PC1). The predictive abilities of models generated with both Google Trends data and conventional COVID-19 metrics were compared with those fitted with conventional COVID-19 metrics only.ResultsThe degree of correlation and the best time lag varied as a function of the selected country and topic searched; in general, the optimal time lag was within 15 days. Overall, predictions of PC1 based on both search terms and COVID-19 traditional metrics performed better than those not including Google searches (median 1.56, IQR 0.90-2.49 versus median 1.87, IQR 1.09-2.95, respectively), but the improvement in prediction varied as a function of the selected country and time frame. The best model varied as a function of country, time range, and period of time selected. Models based on a 7-day moving average led to considerably smaller RMSE values as opposed to those calculated with raw data (median 0.90, IQR 0.50-1.53 versus median 2.27, IQR 1.62-3.74, respectively).ConclusionsThe inclusion of digital online searches in statistical models may improve the nowcasting and forecasting of the COVID-19 epidemic and could be used as one of the surveillance systems of COVID-19 disease. We provide a free web application operating with nearly real-time data that anyone can use to make predictions of outbreaks, improve estimates of the dynamics of ongoing epidemics, and predict future or rebound waves.
| S-EPMC8360333 | biostudies-literature
Project description:BackgroundThe novel coronavirus SARS-CoV-2 and its associated disease, COVID-19, have caused worldwide disruption, leading countries to take drastic measures to address the progression of the disease. As SARS-CoV-2 continues to spread, hospitals are struggling to allocate resources to patients who are most at risk. In this context, it has become important to develop models that can accurately predict the severity of infection of hospitalized patients to help guide triage, planning, and resource allocation.ObjectiveThe aim of this study was to develop accurate models to predict the mortality of hospitalized patients with COVID-19 using basic demographics and easily obtainable laboratory data.MethodsWe performed a retrospective study of 375 hospitalized patients with COVID-19 in Wuhan, China. The patients were randomly split into derivation and validation cohorts. Regularized logistic regression and support vector machine classifiers were trained on the derivation cohort, and accuracy metrics (F1 scores) were computed on the validation cohort. Two types of models were developed: the first type used laboratory findings from the entire length of the patient's hospital stay, and the second type used laboratory findings that were obtained no later than 12 hours after admission. The models were further validated on a multicenter external cohort of 542 patients.ResultsOf the 375 patients with COVID-19, 174 (46.4%) died of the infection. The study cohort was composed of 224/375 men (59.7%) and 151/375 women (40.3%), with a mean age of 58.83 years (SD 16.46). The models developed using data from throughout the patients' length of stay demonstrated accuracies as high as 97%, whereas the models with admission laboratory variables possessed accuracies of up to 93%. The latter models predicted patient outcomes an average of 11.5 days in advance. Key variables such as lactate dehydrogenase, high-sensitivity C-reactive protein, and percentage of lymphocytes in the blood were indicated by the models. In line with previous studies, age was also found to be an important variable in predicting mortality. In particular, the mean age of patients who survived COVID-19 infection (50.23 years, SD 15.02) was significantly lower than the mean age of patients who died of the infection (68.75 years, SD 11.83; P<.001).ConclusionsMachine learning models can be successfully employed to accurately predict outcomes of patients with COVID-19. Our models achieved high accuracies and could predict outcomes more than one week in advance; this promising result suggests that these models can be highly useful for resource allocation in hospitals.
| S-EPMC7572117 | biostudies-literature
Project description:COVID-19 mortality forecasting models provide critical information about the trajectory of the pandemic, which is used by policymakers and public health officials to guide decision-making. However, thousands of published COVID-19 mortality forecasts now exist, many with their own unique methods, assumptions, format, and visualization. As a result, it is difficult to compare models and understand under which circumstances a model performs best. Here, we describe the construction and usability of covidcompare.io, a web tool built to compare numerous forecasts and offer insight into how each has performed over the course of the pandemic. From its launch in December 2020 to June 2021, we have seen 4600 unique visitors from 85 countries. A study conducted with public health professionals showed high usability overall as formally assessed using a Post-Study System Usability Questionnaire. We find that covidcompare.io is an impactful tool for the comparison of international COVID-19 mortality forecasting models.
| S-EPMC8712244 | biostudies-literature
Project description:The need for improved models that can accurately predict COVID-19 dynamics is vital to managing the pandemic and its consequences. We use machine learning techniques to design an adaptive learner that, based on epidemiological data available at any given time, produces a model that accurately forecasts the number of reported COVID-19 deaths and cases in the United States, up to 10 weeks into the future with a mean absolute percentage error of 9%. In addition to being the most accurate long-range COVID predictor so far developed, it captures the observed periodicity in daily reported numbers. Its effectiveness is based on three design features: (1) producing different model parameters to predict the number of COVID deaths (and cases) from each time and for a given number of weeks into the future, (2) systematically searching over the available covariates and their historical values to find an effective combination, and (3) training the model using "last-fold partitioning", where each proposed model is validated on only the last instance of the training dataset, rather than being cross-validated. Assessments against many other published COVID predictors show that this predictor is 19-48% more accurate.
| S-EPMC8257700 | biostudies-literature
Project description:BACKGROUND:COVID-19 is a rapidly emerging respiratory disease caused by SARS-CoV-2. Due to the rapid human-to-human transmission of SARS-CoV-2, many health care systems are at risk of exceeding their health care capacities, in particular in terms of SARS-CoV-2 tests, hospital and intensive care unit (ICU) beds, and mechanical ventilators. Predictive algorithms could potentially ease the strain on health care systems by identifying those who are most likely to receive a positive SARS-CoV-2 test, be hospitalized, or admitted to the ICU. OBJECTIVE:The aim of this study is to develop, study, and evaluate clinical predictive models that estimate, using machine learning and based on routinely collected clinical data, which patients are likely to receive a positive SARS-CoV-2 test or require hospitalization or intensive care. METHODS:Using a systematic approach to model development and optimization, we trained and compared various types of machine learning models, including logistic regression, neural networks, support vector machines, random forests, and gradient boosting. To evaluate the developed models, we performed a retrospective evaluation on demographic, clinical, and blood analysis data from a cohort of 5644 patients. In addition, we determined which clinical features were predictive to what degree for each of the aforementioned clinical tasks using causal explanations. RESULTS:Our experimental results indicate that our predictive models identified patients that test positive for SARS-CoV-2 a priori at a sensitivity of 75% (95% CI 67%-81%) and a specificity of 49% (95% CI 46%-51%), patients who are SARS-CoV-2 positive that require hospitalization with 0.92 area under the receiver operator characteristic curve (AUC; 95% CI 0.81-0.98), and patients who are SARS-CoV-2 positive that require critical care with 0.98 AUC (95% CI 0.95-1.00). CONCLUSIONS:Our results indicate that predictive models trained on routinely collected clinical data could be used to predict clinical pathways for COVID-19 and, therefore, help inform care and prioritize resources.
| S-EPMC7541040 | biostudies-literature
Project description:The disease produced by the new coronavirus known as SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus 2), named COVID-19 (Coronavirus Disease-2019) has recently been classified as a pandemic by the World Health Organization (WHO). However, scarce clinical data is available and generally limited to the Chinese population due to the first cases were identified in Wuhan (Hubei, China).This article describes the rationale and design of the HOPE COVID-19 (Health Outcome Predictive Evaluation for COVID 19) registry (ClinicalTrials.gov Identifier: NCT04334291). With an ambispective cohort design, eligible patients are those discharged, deceased or alive, from any hospital center with a confirmed diagnosis or a COVID-19 high suspicion. With a current recruitment of more than 7000 cases, in 46 hospitals in 8 countries, since it is not possible to estimate the sample size based on literature reports, the investigators will try to get the maximum numbers of patients possible. The study primary objective is all cause mortality and aims to characterize the clinical profile of patients infected in order to develop a prognostic clinical score allowing, rapid logistic decision making. As secondary objectives, the analysis of other clinical events, the risk-adjusted influence of treatments and previous comorbidities of patients infected with the disease will be performed.The results of HOPE COVID-19 will contribute to a better understanding of this condition. We aim to describe the management of this condition as well as the outcomes in relation to the therapy chosen, in order to gain insight into improving patient care in the coming months.Clinical trial registrationClinicalTrials.gov. Unique identifier: NCT04334291.
| S-EPMC7510540 | biostudies-literature
Project description:The COVID-19 pandemic continues to have major impact to health and medical infrastructure, economy, and agriculture. Prominent computational and mathematical models have been unreliable due to the complexity of the spread of infections. Moreover, lack of data collection and reporting makes modelling attempts difficult and unreliable. Hence, we need to re-look at the situation with reliable data sources and innovative forecasting models. Deep learning models such as recurrent neural networks are well suited for modelling spatiotemporal sequences. In this paper, we apply recurrent neural networks such as long short term memory (LSTM), bidirectional LSTM, and encoder-decoder LSTM models for multi-step (short-term) COVID-19 infection forecasting. We select Indian states with COVID-19 hotpots and capture the first (2020) and second (2021) wave of infections and provide two months ahead forecast. Our model predicts that the likelihood of another wave of infections in October and November 2021 is low; however, the authorities need to be vigilant given emerging variants of the virus. The accuracy of the predictions motivate the application of the method in other countries and regions. Nevertheless, the challenges in modelling remain due to the reliability of data and difficulties in capturing factors such as population density, logistics, and social aspects such as culture and lifestyle.
| S-EPMC8797257 | biostudies-literature
Project description: Not available
| S-EPMC8250608 | biostudies-literature
Project description:Hospitals in Canada are facing a crisis-level shortage of critical supplies and equipment during the COVID-19 pandemic. This motivates us to create predictive models that can use Canada COVID-19 data and pandemic-related factors to accurately forecast 5 quantities-three related to hospital resource utilization (i.e., the number of hospital beds, ICU beds, and ventilators that will be needed by COVID-19 patients) and two to the pandemic progress (i.e., the number of COVID-19 cases and COVID-19 deaths)-several weeks in advance. We developed a machine learning method that can use information (i.e., resource utilization, pandemic progress, population mobility, weather condition, and public policy) currently known about a region since March 2020, to learn multiple temporal convolutional network (TCN) models every week; each used for forecasting the weekly average of one of these 5 quantities in Canada (respectively, in six specific provinces) for each, in the next 1 (resp., 2,3,4) weeks. To validate the effectiveness of our method, we compared our method, versus other standard models, on the COVID-19 data and hospital resource data, on the tasks of predicting the 116 values (for Canada and its six most populated provinces), every week from Oct 2020 to July 2021, and the 20 values (only for Canada) for four specific times within 9 July to 31 Dec 2021. Experimental results show that our 4640 TCN models (each forecasting a regional target for a specific future time, on a specific date) can produce accurate 1,2,3,4-week forecasts of the utilization of every hospital resource and pandemic progress for each week from 2 Oct 2020 to 2 July 2021, as well as 80 TCN models for each of the four specified times within 9 July and 31 Dec 2021. Compared to other baseline and state-of-the-art predictive models, our TCN models yielded the best forecasts, with the lowest mean absolute percentage error (MAPE). Additional experiments, on the IHME COVID-19 data, demonstrate the effectiveness of our TCN models, in comparison with IHME forecasts. Each of our TCN models used a pre-defined set of features; we experimentally validate the effectiveness of these features by showing that these models perform better than other models that instead used other features. Overall, these experimental results demonstrate that our method can accurately forecast hospital resource utilization and pandemic progress for Canada and for each of the six provinces.
| S-EPMC9128327 | biostudies-literature
Project description:Timely identification of COVID-19 patients at high risk of mortality can significantly improve patient management and resource allocation within hospitals. This study seeks to develop and validate a data-driven personalized mortality risk calculator for hospitalized COVID-19 patients. De-identified data was obtained for 3,927 COVID-19 positive patients from six independent centers, comprising 33 different hospitals. Demographic, clinical, and laboratory variables were collected at hospital admission. The COVID-19 Mortality Risk (CMR) tool was developed using the XGBoost algorithm to predict mortality. Its discrimination performance was subsequently evaluated on three validation cohorts. The derivation cohort of 3,062 patients has an observed mortality rate of 26.84%. Increased age, decreased oxygen saturation (≤ 93%), elevated levels of C-reactive protein (≥ 130 mg/L), blood urea nitrogen (≥ 18 mg/dL), and blood creatinine (≥ 1.2 mg/dL) were identified as primary risk factors, validating clinical findings. The model obtains out-of-sample AUCs of 0.90 (95% CI, 0.87-0.94) on the derivation cohort. In the validation cohorts, the model obtains AUCs of 0.92 (95% CI, 0.88-0.95) on Seville patients, 0.87 (95% CI, 0.84-0.91) on Hellenic COVID-19 Study Group patients, and 0.81 (95% CI, 0.76-0.85) on Hartford Hospital patients. The CMR tool is available as an online application at covidanalytics.io/mortality_calculator and is currently in clinical use. The CMR model leverages machine learning to generate accurate mortality predictions using commonly available clinical features. This is the first risk score trained and validated on a cohort of COVID-19 patients from Europe and the United States.
| S-EPMC7725386 | biostudies-literature