Dataset Information

Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

ABSTRACT: We present new methodology based on Multi-Objective Markov Decision Processes for developing sequential decision support systems from data. Our approach uses sequential decision-making data to provide support that is useful to many different decision-makers, each with different, potentially time-varying preference. To accomplish this, we develop an extension of fitted-Q iteration for multiple objectives that computes policies for all scalarization functions, i.e. preference functions, simultaneously from continuous-state, finite-horizon data. We identify and address several conceptual and computational challenges along the way, and we introduce a new solution concept that is appropriate when different actions have similar expected outcomes. Finally, we demonstrate an application of our method using data from the Clinical Antipsychotic Trials of Intervention Effectiveness and show that our approach offers decision-makers increased choice by a larger class of optimal policies.

SUBMITTER: Lizotte DJ

PROVIDER: S-EPMC5179144 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

Lizotte Daniel J DJ Laber Eric B EB

Journal of machine learning research : JMLR 20161201

We present new methodology based on Multi-Objective Markov Decision Processes for developing sequential decision support systems from data. Our approach uses sequential decision-making data to provide support that is useful to many different decision-makers, each with different, potentially time-varying preference. To accomplish this, we develop an extension of fitted-<i>Q</i> iteration for multiple objectives that computes policies for all scalarization functions, i.e. preference functions, sim ...[more]

PMID: 28018133

Similar Datasets

Project description:BackgroundMinistries of health in low- and middle-income countries often lack timely quality data for data-driven decision making in healthcare networks. We describe the design and implementation of decision-support electronic tools by the Ministry of Health of the State of Chiapas, in Mexico, as part of Salud Mesoamerica Initiative.MethodsThree electronic decision-support tools were designed through an iterative process focused on streamlined implementation: 1) to collect and report health facility data at health facilities; 2) to compile and analyze data at health district and central level; and, 3) to support stratified sampling of health facilities. Data was collected for five composite indicators measuring availability of equipment, medicines, and supplies for maternal and child health. Quality Assurance Teams collected data, evaluated results and supported quality improvement. Data was also analyzed at the central level and health districts for decision-making.ResultsData from 300 health facilities in four health districts was collected and analyzed (November 2014-June 2015). The first wave revealed gaps on availability of equipment and supplies in more than half of health facilities. Electronic tools provided the ministry of health officers new ways to visualize data, identify patterns and make hypothesis on root-causes. Between the first and second measurement, the number of missing items decreased, and actions performed by quality improvement teams became more proactive. In the final measurement, 89.7-100% of all health facilities achieved all the required items for each indicator.ConclusionsOur experience could help guide others seeking to implement electronic decision-support tools in low- and middle-income countries. Electronic decision-support tools supported data-driven decision-making by identifying gaps on heatmaps and graphs at the health facility, subdistrict, district or state level. Through a rapid improvement process, the Ministry of Health met targets of externally verified indicators. Using available information technology resources facilitated prompt implementation and adoption of technology.

Project description:Objective.Deep brain stimulation (DBS) is an effective treatment for Parkinson's disease (PD) but its success depends on a time-consuming process of trial-and-error to identify the optimal stimulation settings for each individual patient. Data-driven optimization algorithms have been proposed to efficiently find the stimulation setting that maximizes a quantitative biomarker of symptom relief. However, these algorithms cannot efficiently take into account stimulation settings that may control symptoms but also cause side effects. Here we demonstrate how multi-objective data-driven optimization can be used to find the optimal trade-off between maximizing symptom relief and minimizing side effects.Approach.Cortical and motor evoked potential data collected from PD patients during intraoperative stimulation of the subthalamic nucleus were used to construct a framework for designing and prototyping data-driven multi-objective optimization algorithms. Using this framework, we explored how these techniques can be applied clinically, and characterized the design features critical for solving this optimization problem. Our two optimization objectives were to maximize cortical evoked potentials, a putative biomarker of therapeutic benefit, and to minimize motor potentials, a biomarker of motor side effects.Main Results.Using thisin silicodesign framework, we demonstrated how the optimal trade-off between two objectives can substantially reduce the stimulation parameter space by 61 ± 19%. The best algorithm for identifying the optimal trade-off between the two objectives was a Bayesian optimization approach with an area under the receiver operating characteristic curve of up to 0.94 ± 0.02, which was possible with the use of a surrogate model and a well-tuned acquisition function to efficiently select which stimulation settings to sample.Significance.These findings show that multi-objective optimization is a promising approach for identifying the optimal trade-off between symptom relief and side effects in DBS. Moreover, these approaches can be readily extended to newly discovered biomarkers, adapted to DBS for disorders beyond PD, and can scale with the development of more complex DBS devices.

Dataset Information

Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

Publications

Multi-Objective Markov Decision Processes for Data-Driven Decision Support.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets