Dataset Information

Comparison of prediction accuracies between mathematical models to make projections of confirmed cases during the COVID-19 pandamic by country/region.

ABSTRACT:

Background

The COVID-19 pandemic caused >0.228 billion infected cases as of September 18, 2021, implying an exponential growth for infection worldwide. Many mathematical models have been proposed to predict the future cumulative number of infected cases (CNICs). Nevertheless, none compared their prediction accuracies in models. In this work, we compared mathematical models recently published in scholarly journals and designed online dashboards that present actual information about COVID-19.

Methods

All CNICs were downloaded from GitHub. Comparison of model R2 was made in 3 models based on quadratic equation (QE), modified QE (OE-m), and item response theory (IRT) using paired-t test and analysis of variance (ANOVA). The Kano diagram was applied to display the association and the difference in model R2 on a dashboard.

Results

We observed that the correlation coefficient was 0.48 (t = 9.87, n = 265) between QE and IRT models based on R2 when modeling CNICs in a short run (dated from January 1 to February 16, 2021). A significant difference in R2 was found (P < .001, F = 53.32) in mean R2 of 0.98, 0.92, and 0.84 for IRT, OE-mm, and QE, respectively. The IRT-based COVID-19 model is superior to the counterparts of QE-m and QE in model R2 particularly in a longer period of infected days (i.e., in the entire year in 2020).

Conclusion

An online dashboard was demonstrated to display the association and difference in prediction accuracy among predictive models. The IRT mathematical model was recommended to make projections about the evolution of CNICs for each county/region in future applications, not just limited to the COVID-19 epidemic.

SUBMITTER: Tsai KT

PROVIDER: S-EPMC8677971 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:In recent decades, the global incidence of dengue has increased. Affected countries have responded with more effective surveillance strategies to detect outbreaks early, monitor the trends, and implement prevention and control measures. We have applied newly developed machine learning approaches to identify laboratory-confirmed dengue cases from 4,894 emergency department patients with dengue-like illness (DLI) who received laboratory tests. Among them, 60.11% (2942 cases) were confirmed to have dengue. Using just four input variables [age, body temperature, white blood cells counts (WBCs) and platelets], not only the state-of-the-art deep neural network (DNN) prediction models but also the conventional decision tree (DT) and logistic regression (LR) models delivered performances with receiver operating characteristic (ROC) curves areas under curves (AUCs) of the ranging from 83.75% to 85.87% [for DT, DNN and LR: 84.60% ± 0.03%, 85.87% ± 0.54%, 83.75% ± 0.17%, respectively]. Subgroup analyses found all the models were very sensitive particularly in the pre-epidemic period. Pre-peak sensitivities (<35 weeks) were 92.6%, 92.9%, and 93.1% in DT, DNN, and LR respectively. Adjusted odds ratios examined with LR for low WBCs [≤ 3.2 (x103/μL)], fever (≥38°C), low platelet counts [< 100 (x103/μL)], and elderly (≥ 65 years) were 5.17 [95% confidence interval (CI): 3.96-6.76], 3.17 [95%CI: 2.74-3.66], 3.10 [95%CI: 2.44-3.94], and 1.77 [95%CI: 1.50-2.10], respectively. Our prediction models can readily be used in resource-poor countries where viral/serologic tests are inconvenient and can also be applied for real-time syndromic surveillance to monitor trends of dengue cases and even be integrated with mosquito/environment surveillance for early warning and immediate prevention/control measures. In other words, a local community hospital/clinic with an instrument of complete blood counts (including platelets) can provide a sentinel screening during outbreaks. In conclusion, the machine learning approach can facilitate medical and public health efforts to minimize the health threat of dengue epidemics. However, laboratory confirmation remains the primary goal of surveillance and outbreak investigation.

Project description:ObjectivesPost-imaging mathematical prediction models (MPMs) provide guidance for the management of solid pulmonary nodules by providing a lung cancer risk score from demographic and radiologists-indicated imaging characteristics. We hypothesized calibrating the MPM risk score threshold to a local study cohort would result in improved performance over the original recommended MPM thresholds. We compared the pre- and post-calibration performance of four MPM models and determined if improvement in MPM prediction occurs as nodules are imaged longitudinally.Materials and methodsA common cohort of 317 individuals with computed tomography-detected, solid nodules (80 malignant, 237 benign) were used to evaluate the MPM performance. We created a web-based application for this study that allows others to easily calibrate thresholds and analyze the performance of MPMs on their local cohort. Thirty patients with repeated imaging were tested for improved performance longitudinally.ResultsUsing calibrated thresholds, Mayo Clinic and Brock University (BU) MPMs performed the best (AUC = 0.63, 0.61) compared to the Veteran's Affairs (0.51) and Peking University (0.55). Only BU had consensus with the original MPM threshold; the other calibrated thresholds improved MPM accuracy. No significant improvements in accuracy were found longitudinally between time points.ConclusionsCalibration to a common cohort can select the best-performing MPM for your institution. Without calibration, BU has the most stable performance in solid nodules ≥ 8 mm but has only moderate potential to refine subjects into appropriate workup. Application of MPM is recommended only at initial evaluation as no increase in accuracy was achieved over time.Key points• Post-imaging lung cancer risk mathematical predication models (MPMs) perform poorly on local populations without calibration. • An application is provided to facilitate calibration to new study cohorts: the Mayo Clinic model, the U.S. Department of Veteran's Affairs model, the Brock University model, and the Peking University model. • No significant improvement in risk prediction occurred in nodules with repeated imaging sessions, indicating the potential value of risk prediction application is limited to the initial evaluation.