Dataset Information

Human and machine learning in non-Markovian decision making.

ABSTRACT: Humans can learn under a wide variety of feedback conditions. Reinforcement learning (RL), where a series of rewarded decisions must be made, is a particularly important type of learning. Computational and behavioral studies of RL have focused mainly on Markovian decision processes, where the next state depends on only the current state and action. Little is known about non-Markovian decision making, where the next state depends on more than the current state and action. Learning is non-Markovian, for example, when there is no unique mapping between actions and feedback. We have produced a model based on spiking neurons that can handle these non-Markovian conditions by performing policy gradient descent [1]. Here, we examine the model's performance and compare it with human learning and a Bayes optimal reference, which provides an upper-bound on performance. We find that in all cases, our population of spiking neurons model well-describes human performance.

SUBMITTER: Clarke AM

PROVIDER: S-EPMC4405578 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Human and machine learning in non-Markovian decision making.

Clarke Aaron Michael AM Friedrich Johannes J Tartaglia Elisa M EM Marchesotti Silvia S Senn Walter W Herzog Michael H MH

PloS one 20150421 4

Humans can learn under a wide variety of feedback conditions. Reinforcement learning (RL), where a series of rewarded decisions must be made, is a particularly important type of learning. Computational and behavioral studies of RL have focused mainly on Markovian decision processes, where the next state depends on only the current state and action. Little is known about non-Markovian decision making, where the next state depends on more than the current state and action. Learning is non-Markovia ...[more]

PMID: 25898139

Similar Datasets

Project description:BackgroundRepeated invasive intracranial pressure (ICP) monitoring is desirable because many neurosurgical pathologies are associated with elevated ICP. On the other hand, it could become a risk for children to repeat sedation, anesthesia, or radiation exposure. As a non-invasive method, measurements of optic nerve sheath diameter (ONSD) has been revealed to accurately predict increased ICP. However, no studies have indicated a relationship among age, brain, and ventricular parameters in normal children, nor a prediction of increased ICP with artificial intelligence.Methods and findingsThis study enrolled 400 normal children for control and 75 children with signs of increased ICP between 2009 and 2019. Measurements of the parameters including ONSD on CT were obtained. A supervised machine learning was applied to predict suspected increased ICP based on CT measurements. A linear correlation was shown between ln(age) and mean ONSD (mONSD) in normal children, revealing mONSD = 0.36ln(age)+2.26 (R2 = 0.60). This study revealed a linear correlation of mONSD measured on CT with ln(age) and the width of the brain, not the width of the ventricles in 400 normal children based on the univariate analyses. Additionally, the multivariate analyses revealed minimum bicaudate nuclei distance was also associated with mONSD. The results of the group comparison between control and suspected increased ICP revealed a statistical significance in mONSD and the width of the ventricles. The study indicated that supervised machine learning application could be applied to predict suspected increased ICP in children, with an accuracy of 94% for training, 91% for test.ConclusionsThis study clarified three issues regarding ONSD and ICP. Mean ONSD measured on CT was correlated with ln(age) and the width of the brain, not the width of the ventricles in 400 normal children based on the univariate analyses. The multivariate analyses revealed minimum bicaudate nuclei distance was also associated with mONSD. Mean ONSD and the width of ventricles were statistically significant in children with signs of elevated ICP. Finally, the study showed that machine learning could be used to predict children with suspected increased ICP.

Project description:The emergency department (ED) is a fast-paced environment responsible for large volumes of patients with varied disease acuity. Operational pressures on EDs are increasing, which creates the imperative to efficiently identify patients at imminent risk of acute deterioration. The aim of this study is to systematically compare the performance of machine learning algorithms based on logistic regression, gradient boosted decision trees, and support vector machines for predicting imminent clinical deterioration for patients based on cross-sectional patient data extracted from electronic patient records (EPR) at the point of entry to the hospital. We apply state-of-the-art machine learning methods to predict early patient deterioration, based on their first recorded vital signs, observations, laboratory results, and other predictors documented in the EPR. Clinical deterioration in this study is measured by in-hospital mortality and/or admission to critical care. We build on prior work by incorporating interpretable machine learning and fairness-aware modelling, and use a dataset comprising 118, 886 unplanned admissions to Salford Royal Hospital, UK, to systematically compare model variations for predicting mortality and critical care utilisation within 24 hours of admission. We compare model performance to the National Early Warning Score 2 (NEWS2) and yield up to a 0.366 increase in average precision, up to a [Formula: see text] reduction in daily alert rate, and a median 0.599 reduction in differential bias amplification across the protected demographics of age and sex. We use Shapely Additive exPlanations to justify the models' outputs, verify that the captured data associations align with domain knowledge, and pair predictions with the causal context of each patient's most influential characteristics. Introducing our modelling to clinical practice has the potential to reduce alert fatigue and identify high-risk patients with a lower NEWS2 that might be missed currently, but further work is needed to trial the models in clinical practice. We encourage future research to follow a systematised approach to data-driven risk modelling to obtain clinically applicable support tools.

Dataset Information

Human and machine learning in non-Markovian decision making.

Publications

Human and machine learning in non-Markovian decision making.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets