Neural structure mapping in human probabilistic reward learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Luyckx F
PROVIDER: S-EPMC6405242 | biostudies-literature | 2019 Mar
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Predicting future reward is paramount to performing an optimal action. Although a number of brain areas are known to encode such predictions, a detailed account of how the associated representations evolve over time is lacking. Here, we address this question using human magnetoencephalography (MEG) and multivariate analyses of instantaneous activity in reconstructed sources. We overtrained participants on a simple instrumental reward learning task where geometric cues predicted a distribution of possible rewards, from which a sample was revealed 2000 ms later. We show that predicted mean reward (i.e., expected value), and predicted reward variability (i.e., economic risk), are encoded distinctly. Early on, representations of mean reward are seen in parietal and visual areas, and later in frontal regions with orbitofrontal cortex emerging last. Strikingly, an encoding of reward variability emerges simultaneously in parietal/sensory and frontal sources and later than mean reward encoding. An orbitofrontal variability encoding emerged around the same time as that seen for mean reward. Crucially, cross-prediction showed that mean reward and variability representations are distinct and also revealed that instantaneous representations become more stable over time. Across sources, the best fitting metric for variability signals was coefficient of variation (rather than SD or variance), but distinct best metrics were seen for individual brain regions. Our data demonstrate how a dynamic encoding of probabilistic reward prediction unfolds in the brain both in time and space.SIGNIFICANCE STATEMENT Predicting future reward is paramount to optimal behavior. To gain insight into the underlying neural computations, we investigate how reward representations in the brain arise over time. Using magnetoencephalography, we show that a representation of predicted mean reward emerges early in parietal/sensory regions and later in frontal cortex. In contrast, predicted reward variability representations appear in most regions at the same time, and slightly later than for mean reward. For both features, representations dynamically change >1000 ms before stabilizing. The best metric for encoding variability is coefficient of variation, with heterogeneity in this encoding seen between brain areas. The results provide novel insights into the emergence of predictive reward representations.
| S-EPMC5394896 | biostudies-literature
Project description:Bipolar disorder (BPD) features cycling mood states ranging from depression to mania with intermittent phases of euthymia. Bipolar disorder subjects often show excessive goal-directed and pleasure-seeking behavior during manic episodes and reduced hedonic capacity during depressive episodes, indicating that BPD might involve altered reward processing. Our goal was to test the hypothesis that BPD is characterized by impairments in adjusting behavior as a function of prior reinforcement history, particularly in the presence of residual anhedonic symptoms.Eighteen medicated BPD subjects and 25 demographically matched comparison subjects performed a probabilistic reward task. To identify putative dysfunctions in reward processing irrespective of mood state, primary analyses focused on euthymic BPD subjects (n = 13). With signal-detection methodologies, response bias toward a more frequently rewarded stimulus was used to objectively assess the participants' propensity to modulate behavior as a function of reinforcement history.Relative to comparison subjects, euthymic BPD subjects showed a reduced and delayed acquisition of response bias toward the more frequently rewarded stimulus, which was partially due to increased sensitivity to single rewards of the disadvantageous stimulus. Analyses considering the entire BPD sample revealed that reduced reward learning correlated with self-reported anhedonic symptoms, even after adjusting for residual manic and anxious symptoms and general distress.The present study provides preliminary evidence indicating that BPD, even during euthymic states, is characterized by dysfunctional reward learning in situations requiring integration of reinforcement information over time and thus offers initial insights about the potential source of dysfunctional reward processing in this disorder.
| S-EPMC2464620 | biostudies-literature
Project description:The brain represents and reasons probabilistically about complex stimuli and motor actions using a noisy, spike-based neural code. A key building block for such neural computations, as well as the basis for supervised and unsupervised learning, is the ability to estimate the surprise or likelihood of incoming high-dimensional neural activity patterns. Despite progress in statistical modeling of neural responses and deep learning, current approaches either do not scale to large neural populations or cannot be implemented using biologically realistic mechanisms. Inspired by the sparse and random connectivity of real neuronal circuits, we present a model for neural codes that accurately estimates the likelihood of individual spiking patterns and has a straightforward, scalable, efficient, learnable, and realistic neural implementation. This model's performance on simultaneously recorded spiking activity of >100 neurons in the monkey visual and prefrontal cortices is comparable with or better than that of state-of-the-art models. Importantly, the model can be learned using a small number of samples and using a local learning rule that utilizes noise intrinsic to neural circuits. Slower, structural changes in random connectivity, consistent with rewiring and pruning processes, further improve the efficiency and sparseness of the resulting neural representations. Our results merge insights from neuroanatomy, machine learning, and theoretical neuroscience to suggest random sparse connectivity as a key design principle for neuronal computation.
| S-EPMC7547210 | biostudies-literature
Project description:Frontostriatal circuits have been implicated in reward learning, and emerging findings suggest that frontal white matter structural integrity and probabilistic reward learning are reduced in older age. This cross-sectional study examined whether age differences in frontostriatal white matter integrity could account for age differences in reward learning in a community life span sample of human adults. By combining diffusion tensor imaging with a probabilistic reward learning task, we found that older age was associated with decreased reward learning and decreased white matter integrity in specific pathways running from the thalamus to the medial prefrontal cortex and from the medial prefrontal cortex to the ventral striatum. Further, white matter integrity in these thalamocorticostriatal paths could statistically account for age differences in learning. These findings suggest that the integrity of frontostriatal white matter pathways critically supports reward learning. The findings also raise the possibility that interventions that bolster frontostriatal integrity might improve reward learning and decision making.
| S-EPMC3744863 | biostudies-literature
Project description:MicroRNAs (miRNAs) are small regulatory RNAs of approximately 22 nt. Although hundreds of miRNAs have been identified through experimental complementary DNA cloning methods and computational efforts, previous approaches could detect only abundantly expressed miRNAs or close homologs of previously identified miRNAs. Here, we introduce a probabilistic co-learning model for miRNA gene finding, ProMiR, which simultaneously considers the structure and sequence of miRNA precursors (pre-miRNAs). On 5-fold cross-validation with 136 referenced human datasets, the efficiency of the classification shows 73% sensitivity and 96% specificity. When applied to genome screening for novel miRNAs on human chromosomes 16, 17, 18 and 19, ProMiR effectively searches distantly homologous patterns over diverse pre-miRNAs, detecting at least 23 novel miRNA gene candidates. Importantly, the miRNA gene candidates do not demonstrate clear sequence similarity to the known miRNA genes. By quantitative PCR followed by RNA interference against Drosha, we experimentally confirmed that 9 of the 23 representative candidate genes express transcripts that are processed by the miRNA biogenesis enzyme Drosha in HeLa cells, indicating that ProMiR may successfully predict miRNA genes with at least 40% accuracy. Our study suggests that the miRNA gene family may be more abundant than previously anticipated, and confer highly extensive regulatory networks on eukaryotic cells.
| S-EPMC1159118 | biostudies-literature
Project description:Recent research in neuroeconomics has demonstrated that the reinforcement learning model of reward learning captures the patterns of both behavioral performance and neural responses during a range of economic decision-making tasks. However, this powerful theoretical model has its limits. Trial-and-error is only one of the means by which individuals can learn the value associated with different decision options. Humans have also developed efficient, symbolic means of communication for learning without the necessity for committing multiple errors across trials. In the present study, we observed that instructed knowledge of cue-reward probabilities improves behavioral performance and diminishes reinforcement learning-related blood-oxygen level-dependent (BOLD) responses to feedback in the nucleus accumbens, ventromedial prefrontal cortex, and hippocampal complex. The decrease in BOLD responses in these brain regions to reward-feedback signals was functionally correlated with activation of the dorsolateral prefrontal cortex (DLPFC). These results suggest that when learning action values, participants use the DLPFC to dynamically adjust outcome responses in valuation regions depending on the usefulness of action-outcome information.
| S-EPMC3017128 | biostudies-other
Project description:Functional neuroimaging studies of probabilistic category learning in healthy adults report activation of cortical-striatal circuitry. Based on previous findings of normal learning rate concurrent with an overall performance deficit in patients with schizophrenia, we hypothesized that relative to healthy adults, patients with schizophrenia would display preserved caudate nucleus and abnormal prefrontal cortex activation during probabilistic category learning. Forty patients with schizophrenia receiving antipsychotic medication and 25 healthy participants were assessed on interleaved blocks of probabilistic category learning and control tasks while undergoing blood oxygenation level-dependent functional magnetic resonance imaging. In addition to the whole sample of patients with schizophrenia and healthy adults, a subset of patients and healthy adults matched for good learning was also compared. In the whole sample analysis, patients with schizophrenia displayed impaired performance in conjunction with normal learning rate relative to healthy adults. The matched comparison of patients and healthy adults classified as good learners revealed greater caudate and dorsolateral prefrontal cortex activity in the healthy adults and greater activation in a more rostral region of the dorsolateral prefrontal, cingulate, parahippocampal and parietal cortex in patients. These results demonstrate that successful probabilistic category learning can occur in the absence of normal frontal-striatal function. Based on analyses of the patients and healthy adults matched on learning and performance, a minority of patients with schizophrenia achieve successful probabilistic category learning and performance levels through differential activation of a circumscribed neural network which suggests a compensatory mechanism in patients showing successful learning.
| S-EPMC2775494 | biostudies-literature
Project description:From memorizing a musical tune to navigating a well known route, many of our underlying behaviors have a strong temporal component. While the mechanisms behind the sequential nature of the underlying brain activity are likely multifarious and multi-scale, in this work we attempt to characterize to what degree some of this properties can be explained as a consequence of simple associative learning. To this end, we employ a parsimonious firing-rate attractor network equipped with the Hebbian-like Bayesian Confidence Propagating Neural Network (BCPNN) learning rule relying on synaptic traces with asymmetric temporal characteristics. The proposed network model is able to encode and reproduce temporal aspects of the input, and offers internal control of the recall dynamics by gain modulation. We provide an analytical characterisation of the relationship between the structure of the weight matrix, the dynamical network parameters and the temporal aspects of sequence recall. We also present a computational study of the performance of the system under the effects of noise for an extensive region of the parameter space. Finally, we show how the inclusion of modularity in our network structure facilitates the learning and recall of multiple overlapping sequences even in a noisy regime.
| S-EPMC6675053 | biostudies-literature
Project description:Although much work has implicated the contributions of frontostriatal and medial temporal lobe (MTL) systems during probabilistic classification learning, the impact of emotion on these learning circuits is unknown. We used a modified version of the weather prediction task in which two participant groups were scanned with identical neutral cue cards probabilistically linked to either emotional (snake/spider) or neutral (mushroom/flower) outcomes. Owing to the differences in visual information shown as outcomes, analyses were restricted to the cue phase of the trials. Learning rates did not differ between the two groups, although the Emotional group was more likely to use complex strategies and to respond more slowly during initial learning. The Emotional group had reduced frontostriatal and MTL activation relative to the Neutral group, especially for participants who scored higher on snake/spider phobia questionnaires. Accurate performance was more tied to medial prefrontal activity in the Emotional group early in training, and to MTL activity in the Neutral group later in training. Trial-by-trial fluctuations in functional connectivity between the caudate and MTL were also reduced in the Emotional group compared to the Neutral group. Across groups, reaction time indexed a switch in learning systems, with faster trials mediated by the caudate and slower trials mediated by the MTL and frontal lobe. The extent to which the caudate was activated early in training predicted later performance improvements. These results reveal insights into how emotional outcomes modulate procedural learning systems, and the dynamics of MTL-striatal engagement across training trials.
| S-EPMC3195832 | biostudies-literature
Project description:Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.
| S-EPMC5593512 | biostudies-literature