Evaluating short-term forecasting of COVID-19 cases among different epidemiological models under a Bayesian framework.
Ontology highlight
ABSTRACT: Background
Forecasting of COVID-19 cases daily and weekly has been one of the challenges posed to governments and the health sector globally. To facilitate informed public health decisions, the concerned parties rely on short-term daily projections generated via predictive modeling. We calibrate stochastic variants of growth models and the standard susceptible-infectious-removed model into 1 Bayesian framework to evaluate and compare their short-term forecasts.Results
We implement rolling-origin cross-validation to compare the short-term forecasting performance of the stochastic epidemiological models and an autoregressive moving average model across 20 countries that had the most confirmed COVID-19 cases as of August 22, 2020.Conclusion
None of the models proved to be a gold standard across all regions, while all outperformed the autoregressive moving average model in terms of the accuracy of forecast and interpretability.
SUBMITTER: Li Q
PROVIDER: S-EPMC7928884 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The continuously growing number of COVID-19 cases pressures healthcare services worldwide. Accurate short-term forecasting is thus vital to support country-level policy making. The strategies adopted by countries to combat the pandemic vary, generating different uncertainty levels about the actual number of cases. Accounting for the hierarchical structure of the data and accommodating extra-variability is therefore fundamental. We introduce a new modelling framework to describe the pandemic's course with great accuracy and provide short-term daily forecasts for every country in the world. We show that our model generates highly accurate forecasts up to seven days ahead and use estimated model components to cluster countries based on recent events. We introduce statistical novelty in terms of modelling the autoregressive parameter as a function of time, increasing predictive power and flexibility to adapt to each country. Our model can also be used to forecast the number of deaths, study the effects of covariates (such as lockdown policies), and generate forecasts for smaller regions within countries. Consequently, it has substantial implications for global planning and decision making. We present forecasts and make all results freely available to any country in the world through an online Shiny dashboard.
| S-EPMC8024267 | biostudies-literature
Project description:The new Coronavirus (COVID-19) is an emerging disease responsible for infecting millions of people since the first notification until nowadays. Developing efficient short-term forecasting models allow forecasting the number of future cases. In this context, it is possible to develop strategic planning in the public health system to avoid deaths. In this paper, autoregressive integrated moving average (ARIMA), cubist regression (CUBIST), random forest (RF), ridge regression (RIDGE), support vector regression (SVR), and stacking-ensemble learning are evaluated in the task of time series forecasting with one, three, and six-days ahead the COVID-19 cumulative confirmed cases in ten Brazilian states with a high daily incidence. In the stacking-ensemble learning approach, the CUBIST regression, RF, RIDGE, and SVR models are adopted as base-learners and Gaussian process (GP) as meta-learner. The models' effectiveness is evaluated based on the improvement index, mean absolute error, and symmetric mean absolute percentage error criteria. In most of the cases, the SVR and stacking-ensemble learning reach a better performance regarding adopted criteria than compared models. In general, the developed models can generate accurate forecasting, achieving errors in a range of 0.87%-3.51%, 1.02%-5.63%, and 0.95%-6.90% in one, three, and six-days-ahead, respectively. The ranking of models, from the best to the worst regarding accuracy, in all scenarios is SVR, stacking-ensemble learning, ARIMA, CUBIST, RIDGE, and RF models. The use of evaluated models is recommended to forecasting and monitor the ongoing growth of COVID-19 cases, once these models can assist the managers in the decision-making support systems.
| S-EPMC7252162 | biostudies-literature
Project description:BackgroundSimple phenomenological growth models can be useful for estimating transmission parameters and forecasting epidemic trajectories. However, most existing phenomenological growth models only support single-peak outbreak dynamics whereas real epidemics often display more complex transmission trajectories.MethodsWe develop and apply a novel sub-epidemic modeling framework that supports a diversity of epidemic trajectories including stable incidence patterns with sustained or damped oscillations to better understand and forecast epidemic outbreaks. We describe how to forecast an epidemic based on the premise that the observed coarse-scale incidence can be decomposed into overlapping sub-epidemics at finer scales. We evaluate our modeling framework using three outbreak datasets: Severe Acute Respiratory Syndrome (SARS) in Singapore, plague in Madagascar, and the ongoing Ebola outbreak in the Democratic Republic of Congo (DRC) and four performance metrics.ResultsThe sub-epidemic wave model outperforms simpler growth models in short-term forecasts based on performance metrics that account for the uncertainty of the predictions namely the mean interval score (MIS) and the coverage of the 95% prediction interval. For example, we demonstrate how the sub-epidemic wave model successfully captures the 2-peak pattern of the SARS outbreak in Singapore. Moreover, in short-term sequential forecasts, the sub-epidemic model was able to forecast the second surge in case incidence for this outbreak, which was not possible using the simple growth models. Furthermore, our findings support the view that the national incidence curve of the Ebola epidemic in DRC follows a stable incidence pattern with periodic behavior that can be decomposed into overlapping sub-epidemics.ConclusionsOur findings highlight how overlapping sub-epidemics can capture complex epidemic dynamics, including oscillatory behavior in the trajectory of the epidemic wave. This observation has significant implications for interpreting apparent noise in incidence data where the oscillations could be dismissed as a result of overdispersion, rather than an intrinsic part of the epidemic dynamics. Unless the oscillations are appropriately modeled, they could also give a false positive, or negative, impression of the impact from public health interventions. These preliminary results using sub-epidemic models can help guide future efforts to better understand the heterogenous spatial and social factors shaping sub-epidemic patterns for other infectious diseases.
| S-EPMC6704534 | biostudies-literature
Project description:In the immediate aftermath of a strong earthquake and in the presence of an ongoing aftershock sequence, scientific advisories in terms of seismicity forecasts play quite a crucial role in emergency decision-making and risk mitigation. Epidemic Type Aftershock Sequence (ETAS) models are frequently used for forecasting the spatio-temporal evolution of seismicity in the short-term. We propose robust forecasting of seismicity based on ETAS model, by exploiting the link between Bayesian inference and Markov Chain Monte Carlo Simulation. The methodology considers the uncertainty not only in the model parameters, conditioned on the available catalogue of events occurred before the forecasting interval, but also the uncertainty in the sequence of events that are going to happen during the forecasting interval. We demonstrate the methodology by retrospective early forecasting of seismicity associated with the 2016 Amatrice seismic sequence activities in central Italy. We provide robust spatio-temporal short-term seismicity forecasts with various time intervals in the first few days elapsed after each of the three main events within the sequence, which can predict the seismicity within plus/minus two standard deviations from the mean estimate within the few hours elapsed after the main event.
| S-EPMC5575305 | biostudies-literature
Project description:BackgroundMonkeypox virus is gaining attention due to its severity and spread among people. This study sheds light on the modeling and forecasting of new monkeypox cases. Knowledge about the future situation of the virus using a more accurate time series and stochastic models is required for future actions and plans to cope with the challenge.MethodsWe conduct a side-by-side comparison of the machine learning approach with the traditional time series model. The multilayer perceptron model (MLP), a machine learning technique, and the Box-Jenkins methodology, also known as the ARIMA model, are used for classical modeling. Both methods are applied to the Monkeypox cumulative data set and compared using different model selection criteria such as root mean square error, mean square error, mean absolute error, and mean absolute percentage error.ResultsWith a root mean square error of 150.78, the monkeypox series follows the ARIMA (7,1,7) model among the other potential models. Comparatively, we use the multilayer perceptron (MLP) model, which employs the sigmoid activation function and has a different number of hidden neurons in a single hidden layer. The root mean square error of the MLP model, which uses a single input and ten hidden neurons, is 54.40, significantly lower than that of the ARIMA model. The actual confirmed cases versus estimated or fitted plots also demonstrate that the multilayer perceptron model has a better fit for the monkeypox data than the ARIMA model.Conclusions and recommendationWhen it comes to predicting monkeypox, the machine learning method outperforms the traditional time series. A better match can be achieved in future studies by applying the extreme learning machine model (ELM), support vector machine (SVM), and some other methods with various activation functions. It is thus concluded that the selected data provide a real picture of the virus. If the situations remain the same, governments and other stockholders should ensure the follow-up of Standard Operating Procedures (SOPs) among the masses, as the trends will continue rising in the upcoming 10 days. However, governments should take some serious interventions to cope with the virus.LimitationIn the ARIMA models selected for forecasting, we did not incorporate the effect of covariates such as the effect of net migration of monkeypox virus patients, government interventions, etc.
| S-EPMC9659136 | biostudies-literature
Project description:BackgroundThe WHO announced the epidemic of SARS-CoV2 as a public health emergency of international concern on 30th January 2020. To date, it has spread to more than 200 countries and has been declared a global pandemic. For appropriate preparedness, containment, and mitigation response, the stakeholders and policymakers require prior guidance on the propagation of SARS-CoV2.MethodologyThis study aims to provide such guidance by forecasting the cumulative COVID-19 cases up to 4 weeks ahead for 187 countries, using four data-driven methodologies; autoregressive integrated moving average (ARIMA), exponential smoothing model (ETS), and random walk forecasts (RWF) with and without drift. For these forecasts, we evaluate the accuracy and systematic errors using the Mean Absolute Percentage Error (MAPE) and Mean Absolute Error (MAE), respectively.FindingsThe results show that the ARIMA and ETS methods outperform the other two forecasting methods. Additionally, using these forecasts, we generate heat maps to provide a pictorial representation of the countries at risk of having an increase in the cases in the coming 4 weeks of February 2021.ConclusionDue to limited data availability during the ongoing pandemic, less data-hungry short-term forecasting models, like ARIMA and ETS, can help in anticipating the future outbreaks of SARS-CoV2.
| S-EPMC8139504 | biostudies-literature
Project description:Modeling time series has been a particularly challenging aspect due to the need for constant adjustments in a rapidly changing environment, data uncertainty, dependencies between variables, volatile fluctuations, and the need to identify ideal hyperparameters. The present study presents a Framework capable of making projections from time series related to cases and deaths by COVID-19 in the Amazonian state of Pará, in Brazil. For the first time, deep learning models such as TCN, TRANSFORMER, TFT, N-BEATS, and N-HiTS were assessed for this purpose. The ARIMA statistical model was also used in post-processing for residual adjustment and short-term smoothing of the generated forecasts. The Framework generates probabilistic forecasts, with multivariate support, considering the following variables: daily cases per day of the first symptom, cases published daily, the occurrence of deaths, deaths published daily, and percentage of daily vaccination. The generated predictions are statistically evaluated by determining the best model for 7-day moving average projections using evaluating metrics such as MSE, RMSE, MAPE, sMAPE, r2, Coefficient of Variation, and residual analysis. As a result, the generated projections showed an average error of 5.4% for Cases Publication, 8.0% for Cases Symptoms, 11.12% for Deaths Publication, and 4.6% for Deaths Occurrence, with the N-HiTS and N-BEATS models obtaining better results. In general terms, the use of deep learning models to predict cases and deaths from COVID-19 has proven to be a valuable practice for analyzing the spread of the virus, which allows health managers to better understand and respond to this kind of pandemic outbreak.
| S-EPMC10656034 | biostudies-literature
Project description:BACKGROUND:Trachoma programs rely on guidelines made in large part using expert opinion of what will happen with and without intervention. Large community-randomized trials offer an opportunity to actually compare forecasting methods in a masked fashion. METHODS:The Program for the Rapid Elimination of Trachoma trials estimated longitudinal prevalence of ocular chlamydial infection from 24 communities treated annually with mass azithromycin. Given antibiotic coverage and biannual assessments from baseline through 30 months, forecasts of the prevalence of infection in each of the 24 communities at 36 months were made by three methods: the sum of 15 experts' opinion, statistical regression of the square-root-transformed prevalence, and a stochastic hidden Markov model of infection transmission (Susceptible-Infectious-Susceptible, or SIS model). All forecasters were masked to the 36-month results and to the other forecasts. Forecasts of the 24 communities were scored by the likelihood of the observed results and compared using Wilcoxon's signed-rank statistic. FINDINGS:Regression and SIS hidden Markov models had significantly better likelihood than community expert opinion (p = 0.004 and p = 0.01, respectively). All forecasts scored better when perturbed to decrease Fisher's information. Each individual expert's forecast was poorer than the sum of experts. INTERPRETATION:Regression and SIS models performed significantly better than expert opinion, although all forecasts were overly confident. Further model refinements may score better, although would need to be tested and compared in new masked studies. Construction of guidelines that rely on forecasting future prevalence could consider use of mathematical and statistical models. TRIAL REGISTRATION:Clinicaltrials.gov NCT00792922.
| S-EPMC4547743 | biostudies-literature
Project description:The COVID-19 has caused the deadliest pandemic around the globe, emerged from the city of Wuhan, China by the end of 2019 and affected all continents of the world, with severe health implications and as well as financial-damage. Pakistan is also amongst the top badly effected countries in terms of casualties and financial loss due to COVID-19. By 20th March, 2021, Pakistan reported 623,135 total confirmed cases and 13,799 deaths. A state space model called 'Bayesian Dynamic Linear Model' (BDLM) was used for the forecast of daily new infections, deaths and recover cases regarding COVID-19. For the estimation of states of the models and forecasting new observations, the recursive Kalman filter was used. Twenty days ahead forecast show that the maximum number of new infections are 4,031 per day with 95% prediction interval (3,319-4,743). Death forecast shows that the maximum number of the deaths with 95% prediction interval are 81 and (67-93), respectively. Maximum daily recoveries are 3,464 with 95% prediction interval (2,887-5,423) in the next 20 days. The average number of new infections, deaths and recover cases are 3,282, 52 and 1,840, respectively, in the upcoming 20 days. As the data generation processes based on the latest data has been identified, therefore it can be updated with the availability of new data to provide latest forecast.
| S-EPMC8211153 | biostudies-literature
Project description:ObjectiveTo illustrate the benefits of deploying individual headache forecasting models using continuous updating with Bayesian methods.BackgroundThe ability to reliably forecast headache attacks within an individual over time would enhance the study of attacks and allow preemptive treatment. However, deploying a suitable forecasting model in a clinical setting will likely involve several unique challenges related to heterogeneity in the predictor weights, limited or sparse data, and the need for a quick "warm-up." The use of Bayesian methods offers solutions to each of these specific challenges.MethodsThis was a post hoc analysis of a cohort study of individuals with episodic migraine attacks. Individuals completed daily diaries that allowed the estimation of several forecasting models, each using different types of ancillary information incorporated into formal prior probability distributions. An in silico analysis was conducted that mimicked the deployment of these models in a clinical-like setting where the parameters of the models were continuously updated and evaluated each day using root mean square error (RMSE).ResultsIndividuals (N = 95) were followed for 50 days and contributed 3359 days of nonmissing diary data. During the observation period, there were 1293/3359 (38.5%) days with a headache attack. Self-reported baseline headache frequency was associated with the corresponding predicted probability of future attacks, r = 0.15-0.39. At Day 25, the correlation between baseline information and predicted attack likelihood was r = 0.29 (95% CI: 0.09-0.47). Additionally, the use of prior probability distributions for model parameters improved the model fit, especially early in the deployment of the models (e.g., Day 5 RMSE 0.45 vs. 0.43). Models using informative prior probability distributions outperformed the models estimated without this information during the first 42 days of observation, although performance became more similar as more data were collected.ConclusionsThis analysis demonstrates the value of Bayesian methods in using additional available information to improve forecasting model performance, especially early in the deployment of a forecasting model. To obtain the full value of such models or to apply any model in clinical settings, a model with adequate discrimination and calibration will be needed.
| S-EPMC8672809 | biostudies-literature