Dataset Information

An open challenge to advance probabilistic forecasting for dengue epidemics.

ABSTRACT: A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global public health problem. Sixteen teams used a variety of methods and data to generate forecasts for 3 epidemiological targets (peak incidence, the week of the peak, and total incidence) over 8 dengue seasons in Iquitos, Peru and San Juan, Puerto Rico. Forecast skill was highly variable across teams and targets. While numerous forecasts showed high skill for midseason situational awareness, early season skill was low, and skill was generally lowest for high incidence seasons, those for which forecasts would be most valuable. A comparison of modeling approaches revealed that average forecast skill was lower for models including biologically meaningful data and mechanisms and that both multimodel and multiteam ensemble forecasts consistently outperformed individual model forecasts. Leveraging these insights, data, and the forecasting framework will be critical to improve forecast skill and the application of forecasts in real time for epidemic preparedness and response. Moreover, key components of this project-integration with public health needs, a common forecasting framework, shared and standardized data, and open participation-can help advance infectious disease forecasting beyond dengue.

SUBMITTER: Johansson MA

PROVIDER: S-EPMC6883829 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An open challenge to advance probabilistic forecasting for dengue epidemics.

Johansson Michael A MA Apfeldorf Karyn M KM Dobson Scott S Devita Jason J Buczak Anna L AL Baugher Benjamin B Moniz Linda J LJ Bagley Thomas T Babin Steven M SM Guven Erhan E Yamana Teresa K TK Shaman Jeffrey J Moschou Terry T Lothian Nick N Lane Aaron A Osborne Grant G Jiang Gao G Brooks Logan C LC Farrow David C DC Hyun Sangwon S Tibshirani Ryan J RJ Rosenfeld Roni R Lessler Justin J Reich Nicholas G NG Cummings Derek A T DAT Lauer Stephen A SA Moore Sean M SM Clapham Hannah E HE Lowe Rachel R Bailey Trevor C TC García-Díez Markel M Carvalho Marilia Sá MS Rodó Xavier X Sardar Tridip T Paul Richard R Ray Evan L EL Sakrejda Krzysztof K Brown Alexandria C AC Meng Xi X Osoba Osonde O Vardavas Raffaele R Manheim David D Moore Melinda M Rao Dhananjai M DM Porco Travis C TC Ackley Sarah S Liu Fengchen F Worden Lee L Convertino Matteo M Liu Yang Y Reddy Abraham A Ortiz Eloy E Rivero Jorge J Brito Humberto H Juarrero Alicia A Johnson Leah R LR Gramacy Robert B RB Cohen Jeremy M JM Mordecai Erin A EA Murdock Courtney C CC Rohr Jason R JR Ryan Sadie J SJ Stewart-Ibarra Anna M AM Weikel Daniel P DP Jutla Antarpreet A Khan Rakibul R Poultney Marissa M Colwell Rita R RR Rivera-García Brenda B Barker Christopher M CM Bell Jesse E JE Biggerstaff Matthew M Swerdlow David D Mier-Y-Teran-Romero Luis L Forshey Brett M BM Trtanj Juli J Asher Jason J Clay Matt M Margolis Harold S HS Hebbeler Andrew M AM George Dylan D Chretien Jean-Paul JP

Proceedings of the National Academy of Sciences of the United States of America 20191111 48

A wide range of research has promised new tools for forecasting infectious disease dynamics, but little of that research is currently being applied in practice, because tools do not address key public health needs, do not produce probabilistic forecasts, have not been evaluated on external data, or do not provide sufficient forecast skill to be useful. We developed an open collaborative forecasting challenge to assess probabilistic forecasts for seasonal epidemics of dengue, a major global publi ...[more]

PMID: 31712420

Similar Datasets

Project description:BackgroundDengue dynamics are driven by complex interactions between human-hosts, mosquito-vectors and viruses that are influenced by environmental and climatic factors. The objectives of this study were to analyze and model the relationships between climate, Aedes aegypti vectors and dengue outbreaks in Noumea (New Caledonia), and to provide an early warning system.Methodology/principal findingsEpidemiological and meteorological data were analyzed from 1971 to 2010 in Noumea. Entomological surveillance indices were available from March 2000 to December 2009. During epidemic years, the distribution of dengue cases was highly seasonal. The epidemic peak (March-April) lagged the warmest temperature by 1-2 months and was in phase with maximum precipitations, relative humidity and entomological indices. Significant inter-annual correlations were observed between the risk of outbreak and summertime temperature, precipitations or relative humidity but not ENSO. Climate-based multivariate non-linear models were developed to estimate the yearly risk of dengue outbreak in Noumea. The best explicative meteorological variables were the number of days with maximal temperature exceeding 32°C during January-February-March and the number of days with maximal relative humidity exceeding 95% during January. The best predictive variables were the maximal temperature in December and maximal relative humidity during October-November-December of the previous year. For a probability of dengue outbreak above 65% in leave-one-out cross validation, the explicative model predicted 94% of the epidemic years and 79% of the non epidemic years, and the predictive model 79% and 65%, respectively.Conclusions/significanceThe epidemic dynamics of dengue in Noumea were essentially driven by climate during the last forty years. Specific conditions based on maximal temperature and relative humidity thresholds were determinant in outbreaks occurrence. Their persistence was also crucial. An operational model that will enable health authorities to anticipate the outbreak risk was successfully developed. Similar models may be developed to improve dengue management in other countries.

Project description:BackgroundWith enough advanced notice, dengue outbreaks can be mitigated. As a climate-sensitive disease, environmental conditions and past patterns of dengue can be used to make predictions about future outbreak risk. These predictions improve public health planning and decision-making to ultimately reduce the burden of disease. Past approaches to dengue forecasting have used seasonal climate forecasts, but the predictive ability of a system using different lead times in a year-round prediction system has been seldom explored. Moreover, the transition from theoretical to operational systems integrated with disease control activities is rare.Methods and findingsWe introduce an operational seasonal dengue forecasting system for Vietnam where Earth observations, seasonal climate forecasts, and lagged dengue cases are used to drive a superensemble of probabilistic dengue models to predict dengue risk up to 6 months ahead. Bayesian spatiotemporal models were fit to 19 years (2002-2020) of dengue data at the province level across Vietnam. A superensemble of these models then makes probabilistic predictions of dengue incidence at various future time points aligned with key Vietnamese decision and planning deadlines. We demonstrate that the superensemble generates more accurate predictions of dengue incidence than the individual models it incorporates across a suite of time horizons and transmission settings. Using historical data, the superensemble made slightly more accurate predictions (continuous rank probability score [CRPS] = 66.8, 95% CI 60.6-148.0) than a baseline model which forecasts the same incidence rate every month (CRPS = 79.4, 95% CI 78.5-80.5) at lead times of 1 to 3 months, albeit with larger uncertainty. The outbreak detection capability of the superensemble was considerably larger (69%) than that of the baseline model (54.5%). Predictions were most accurate in southern Vietnam, an area that experiences semi-regular seasonal dengue transmission. The system also demonstrated added value across multiple areas compared to previous practice of not using a forecast. We use the system to make a prospective prediction for dengue incidence in Vietnam for the period May to October 2020. Prospective predictions made with the superensemble were slightly more accurate (CRPS = 110, 95% CI 102-575) than those made with the baseline model (CRPS = 125, 95% CI 120-168) but had larger uncertainty. Finally, we propose a framework for the evaluation of probabilistic predictions. Despite the demonstrated value of our forecasting system, the approach is limited by the consistency of the dengue case data, as well as the lack of publicly available, continuous, and long-term data sets on mosquito control efforts and serotype-specific case data.ConclusionsThis study shows that by combining detailed Earth observation data, seasonal climate forecasts, and state-of-the-art models, dengue outbreaks can be predicted across a broad range of settings, with enough lead time to meaningfully inform dengue control. While our system omits some important variables not currently available at a subnational scale, the majority of past outbreaks could be predicted up to 3 months ahead. Over the next 2 years, the system will be prospectively evaluated and, if successful, potentially extended to other areas and other climate-sensitive disease systems.

Project description:BackgroundWest Nile virus (WNV) is the leading cause of mosquito-borne illness in the continental USA. WNV occurrence has high spatiotemporal variation, and current approaches to targeted control of the virus are limited, making forecasting a public health priority. However, little research has been done to compare strengths and weaknesses of WNV disease forecasting approaches on the national scale. We used forecasts submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement.MethodsWe performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. In the evaluation, we included forecasts produced by comparison models of varying complexity as benchmarks of forecast performance. We also used regression analysis to identify modeling approaches and contextual factors that were associated with forecast skill.ResultsSimple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Forecast skill improved across updated forecast submissions submitted during the 2020 season. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. We also identified population size, extreme minimum winter temperature, and interannual variation in WNND cases as county-level characteristics associated with variation in forecast skill.ConclusionsHistorical WNND cases were strong predictors of future cases with minimal increase in skill achieved by models that included other factors. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Also, the prediction of outbreaks, which are outliers relative to typical case numbers, remains difficult. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g. current weather and preliminary human cases).

Dataset Information

An open challenge to advance probabilistic forecasting for dengue epidemics.

Publications

An open challenge to advance probabilistic forecasting for dengue epidemics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets