Dataset Information

A Multiarmed Bandit Approach to Adaptive Water Quality Management.

ABSTRACT: Nonpoint source water quality management is challenged with allocating uncertain management actions and monitoring their performance in the absence of state-dependent decision making. This adaptive management context can be expressed as a multiarmed bandit problem. Multiarmed bandit strategies attempt to balance the exploitation of actions that appear to maximize performance with the exploration of uncertain, but potentially better, actions. We performed a test of multiarmed bandit strategies to inform adaptive water quality management in Massachusetts, USA. Conservation and restoration practitioners were tasked with allocating household wastewater treatments to minimize N inputs to impaired waters. We obtained time series of N monitoring data from 3 wastewater treatment types and organized them chronologically and randomly. The chronological data set represented nonstationary performance based on recent monitoring data, whereas the random data set represented stationary performance. We tested 2 multiarmed bandit strategies in hypothetical experiments to sample from the treatment data through 20 sequential decisions. A deterministic probability-matching strategy allocated treatments with the highest probability of success regarding their performance at each decision. A randomized probability-matching strategy randomly allocated treatments according to their probability of success at each decision. The strategies were compared with a nonadaptive strategy that equally allocated treatments at each decision. Results indicated that equal allocation is useful for learning in nonstationary situations but tended to overexplore inferior treatments and thus did not maximize performance when compared with the other strategies. Deterministic probability matching maximized performance in many stationary situations, but the strategy did not adequately explore treatments and converged on inferior treatments in nonstationary situations. Randomized probability matching balanced performance and learning in stationary situations, but the strategy could converge on inferior treatments in nonstationary situations. These findings provide evidence that probability-matching strategies are useful for adaptive management. Integr Environ Assess Manag 2020;00:1-12. © 2020 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC).

SUBMITTER: Martin DM

PROVIDER: S-EPMC7689691 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Multiarmed Bandit Approach to Adaptive Water Quality Management.

Martin David M DM Johnson Fred A FA

Integrated environmental assessment and management 20200814 6

Nonpoint source water quality management is challenged with allocating uncertain management actions and monitoring their performance in the absence of state-dependent decision making. This adaptive management context can be expressed as a multiarmed bandit problem. Multiarmed bandit strategies attempt to balance the exploitation of actions that appear to maximize performance with the exploration of uncertain, but potentially better, actions. We performed a test of multiarmed bandit strategies to ...[more]

PMID: 32584467

Similar Datasets

Project description:In densely populated areas, surface waters are affected by many sources of pollution. Besides classical pollutants like nutrients and organic matter that lead to eutrophication, micropollutants from various point- and non-point sources are getting more attention by water quality managers. For cost-effective management an integrated assessment is needed that takes into account all relevant pollutants and all sources of pollution within a catchment. Due to the difficulty of identifying and quantifying sources of pollution and the need for considering long-term changes in boundary conditions, typically substantial uncertainty exists about the consequences of potential management alternatives to improve surface water quality. We therefore need integrated assessment methods that are able to deal with multiple objectives and account for various sources of uncertainty. This paper aims to contribute to integrated, prospective water management by combining a) multi-criteria decision support methods to structure the decision process and quantify preferences, b) integrated water quality modelling to predict consequences of management alternatives accounting for uncertainty, and c) scenario planning to consider uncertainty from potential future climate and socio-economic developments, to evaluate the future cost-effectiveness of water quality management alternatives at the catchment scale. It aims to demonstrate the usefulness of multi-attribute value functions for water quality assessment to i) propagate uncertainties throughout the entire assessment procedure, ii) facilitate the aggregation of multiple objectives while avoiding discretization errors when using categories for sub-objectives, iii) transparently communicate the results. We show how to use such multi-attribute value functions for model-based decision support in water quality management. We showcase the procedure for the Mönchaltorfer Aa catchment on the Swiss Plateau. We evaluate ten different water quality management alternatives, including current practice, that tackle macro- and micropollutants from a wide spectrum of agricultural and urban sources. We evaluate costs and water quality effects of the alternatives under four different socio-economic scenarios for the horizon 2050 under present and future climate projections and visualize their uncertainty. While the performance of alternatives is catchment specific, the methods can be transferred to other places and other management situations. Results confirm the need for cross-sectoral coordination of different management actions and interdisciplinary collaboration to support the development of prospective strategies to improve water quality.

Dataset Information

A Multiarmed Bandit Approach to Adaptive Water Quality Management.

Publications

A Multiarmed Bandit Approach to Adaptive Water Quality Management.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets