Dataset Information

Partitioning prediction uncertainty in climate-dependent population models.

ABSTRACT: The science of complex systems is increasingly asked to forecast the consequences of climate change. As a result, scientists are now engaged in making predictions about an uncertain future, which entails the efficient communication of this uncertainty. Here we show the benefits of hierarchically decomposing the uncertainty in predicted changes in animal population size into its components due to structural uncertainty in climate scenarios (greenhouse gas emissions and global circulation models), structural uncertainty in the demographic model, climatic stochasticity, environmental stochasticity unexplained by climate-demographic trait relationships, and sampling variance in demographic parameter estimates. We quantify components of uncertainty surrounding the future abundance of a migratory bird, the greater snow goose (Chen caeruslescens atlantica), using a process-based demographic model covering their full annual cycle. Our model predicts a slow population increase but with a large prediction uncertainty. As expected from theoretical variance decomposition rules, the contribution of sampling variance to prediction uncertainty rapidly overcomes that of process variance and dominates. Among the sources of process variance, uncertainty in the climate scenarios contributed less than 3% of the total prediction variance over a 40-year period, much less than environmental stochasticity. Our study exemplifies opportunities to improve the forecasting of complex systems using long-term studies and the challenges inherent to predicting the future of stochastic systems.

SUBMITTER: Gauthier G

PROVIDER: S-EPMC5204174 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Partitioning prediction uncertainty in climate-dependent population models.

Gauthier Gilles G Péron Guillaume G Lebreton Jean-Dominique JD Grenier Patrick P van Oudenhove Louise L

Proceedings. Biological sciences 20161201 1845

The science of complex systems is increasingly asked to forecast the consequences of climate change. As a result, scientists are now engaged in making predictions about an uncertain future, which entails the efficient communication of this uncertainty. Here we show the benefits of hierarchically decomposing the uncertainty in predicted changes in animal population size into its components due to structural uncertainty in climate scenarios (greenhouse gas emissions and global circulation models), ...[more]

PMID: 28003456

Similar Datasets

Project description:Near-term ecological forecasts provide resource managers advance notice of changes in ecosystem services, such as fisheries stocks, timber yields, or water quality. Importantly, ecological forecasts can identify where there is uncertainty in the forecasting system, which is necessary to improve forecast skill and guide interpretation of forecast results. Uncertainty partitioning identifies the relative contributions to total forecast variance introduced by different sources, including specification of the model structure, errors in driver data, and estimation of current states (initial conditions). Uncertainty partitioning could be particularly useful in improving forecasts of highly variable cyanobacterial densities, which are difficult to predict and present a persistent challenge for lake managers. As cyanobacteria can produce toxic and unsightly surface scums, advance warning when cyanobacterial densities are increasing could help managers mitigate water quality issues. Here, we fit 13 Bayesian state-space models to evaluate different hypotheses about cyanobacterial densities in a low nutrient lake that experiences sporadic surface scums of the toxin-producing cyanobacterium, Gloeotrichia echinulata. We used data from several summers of weekly cyanobacteria samples to identify dominant sources of uncertainty for near-term (1- to 4-week) forecasts of G. echinulata densities. Water temperature was an important predictor of cyanobacterial densities during model fitting and at the 4-week forecast horizon. However, no physical covariates improved model performance over a simple model including the previous week's densities in 1-week-ahead forecasts. Even the best fit models exhibited large variance in forecasted cyanobacterial densities and did not capture rare peak occurrences, indicating that significant explanatory variables when fitting models to historical data are not always effective for forecasting. Uncertainty partitioning revealed that model process specification and initial conditions dominated forecast uncertainty. These findings indicate that long-term studies of different cyanobacterial life stages and movement in the water column as well as measurements of drivers relevant to different life stages could improve model process representation of cyanobacteria abundance. In addition, improved observation protocols could better define initial conditions and reduce spatial misalignment of environmental data and cyanobacteria observations. Our results emphasize the importance of ecological forecasting principles and uncertainty partitioning to refine and understand predictive capacity across ecosystems.

Project description:Scientists agree that the climate is changing due to human activities, but there is less agreement about the specific consequences and their timeline. Disagreement among climate projections is attributable to the complexity of climate models that differ in their structure, parameters, initial conditions, etc. We examine how different sources of uncertainty affect people's interpretation of, and reaction to, information about climate change by presenting participants forecasts from multiple experts. Participants viewed three types of sets of sea-level rise projections: (1) precise, but conflicting; (2) imprecise, but agreeing, and (3) hybrid that were both conflicting and imprecise. They estimated the most likely sea-level rise, provided a range of possible values and rated the sets on several features - ambiguity, credibility, completeness, etc. In Study 1, everyone saw the same hybrid set. We found that participants were sensitive to uncertainty between sources, but not to uncertainty about which model was used. The impacts of conflict and imprecision were combined for estimation tasks and compromised for feature ratings. Estimates were closer to the experts' original projections, and sets were rated more favorably under imprecision. Estimates were least consistent with (narrower than) the experts in the hybrid condition, but participants rated the conflicting set least favorably. In Study 2, we investigated the hybrid case in more detail by creating several distinct interval sets that combine conflict and imprecision. Two factors drive perceptual differences: overlap - the structure of the forecast set (whether intersecting, nested, tangent, or disjoint) - and asymmetry - the balance of the set. Estimates were primarily driven by asymmetry, and preferences were primarily driven by overlap. Asymmetric sets were least consistent with the experts: estimated ranges were narrower, and estimates of the most likely value were shifted further below the set mean. Intersecting and nested sets were rated similarly to imprecision, and ratings of disjoint and tangent sets were rated like conflict. Our goal was to determine which underlying factors of information sets drive perceptions of uncertainty in consistent, predictable ways. The two studies lead us to conclude that perceptions of agreement require intersection and balance, and overly precise forecasts lead to greater perceptions of disagreement and a greater likelihood of the public discrediting and misinterpreting information.

Project description:Developmental models, such as degree-day models, are commonly used to predict the impact of future climate change on the intensity, distribution, and timing of the transmission of infectious diseases, particularly those caused by pathogens carried by vectors or intermediate hosts. Resulting projections can be useful in policy discussions concerning regional or national responses to future distributions of important infectious diseases. Although the simplicity of degree-day models is appealing, little work has been done to analyze their ability to make reliable projections of the distribution of important pathogens, vectors, or intermediate hosts in the presence of the often considerable parametric uncertainty common to such models. Here, a population model of Oncomelania hupensis, the intermediate host of Schistosoma japonicum, was used to investigate the sensitivity of host range predictions in Sichuan Province, China, to uncertainty in two key degree-day model parameters: delta(min) (minimum temperature threshold for development) and K (total degree-days required for completion of snail development). The intent was to examine the consequences of parametric uncertainty in a plausible biological model, rather than to generate the definitive model. Results indicate that model output, the seasonality of population dynamics, and range predictions, particularly along the edge of the range, are highly sensitive to changes in model parameters, even at levels of parametric uncertainty common to such applications. Caution should be used when interpreting the results of degree-day models used to generate predictions of disease distribution and risk under scenarios of future climate change, and predictions should be considered most reliable when the temperature ranges used in projections resemble those used to estimate model parameters. Given the potential for substantial changes in degree-day model output with modest changes in parameter values, caution is warranted when results will be used to inform policy and management decisions.

Project description:BackgroundGenerally, QSAR modelling requires both model selection and validation since there is no a priori knowledge about the optimal QSAR model. Prediction errors (PE) are frequently used to select and to assess the models under study. Reliable estimation of prediction errors is challenging - especially under model uncertainty - and requires independent test objects. These test objects must not be involved in model building nor in model selection. Double cross-validation, sometimes also termed nested cross-validation, offers an attractive possibility to generate test data and to select QSAR models since it uses the data very efficiently. Nevertheless, there is a controversy in the literature with respect to the reliability of double cross-validation under model uncertainty. Moreover, systematic studies investigating the adequate parameterization of double cross-validation are still missing. Here, the cross-validation design in the inner loop and the influence of the test set size in the outer loop is systematically studied for regression models in combination with variable selection.MethodsSimulated and real data are analysed with double cross-validation to identify important factors for the resulting model quality. For the simulated data, a bias-variance decomposition is provided.ResultsThe prediction errors of QSAR/QSPR regression models in combination with variable selection depend to a large degree on the parameterization of double cross-validation. While the parameters for the inner loop of double cross-validation mainly influence bias and variance of the resulting models, the parameters for the outer loop mainly influence the variability of the resulting prediction error estimate.ConclusionsDouble cross-validation reliably and unbiasedly estimates prediction errors under model uncertainty for regression models. As compared to a single test set, double cross-validation provided a more realistic picture of model quality and should be preferred over a single test set.

Dataset Information

Partitioning prediction uncertainty in climate-dependent population models.

Publications

Partitioning prediction uncertainty in climate-dependent population models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets