Closed-Loop Theta Stimulation in the Orbitofrontal Cortex Prevents Reward-Based Learning.
Ontology highlight
ABSTRACT:
SUBMITTER: Knudsen EB
PROVIDER: S-EPMC7480400 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Humans and other primates can reverse their choice of stimuli in one trial when the rewards delivered by the stimuli change or reverse. Rapidly changing our behavior when the rewards change is important for many types of behavior, including emotional and social behavior. It is shown in a one-trial rule-based Go-NoGo deterministic visual discrimination reversal task to obtain points, that the human right lateral orbitofrontal cortex and adjoining inferior frontal gyrus is activated on reversal trials, when an expected reward is not obtained, and the non-reward allows the human to switch choices based on a rule. This reward reversal goes beyond model-free reinforcement learning. This functionality of the right lateral orbitofrontal cortex shown here in very rapid, one-trial, rule-based changes in human behavior when a reward is not received is related to the emotional and social changes that follow orbitofrontal cortex damage, and to depression in which this non-reward system is oversensitive and over-connected.
| S-EPMC8152898 | biostudies-literature
Project description:Closed-loop brain stimulation refers to capturing neurophysiological measures such as electroencephalography (EEG), quickly identifying neural events of interest, and producing auditory, magnetic or electrical stimulation so as to interact with brain processes precisely. It is a promising new method for fundamental neuroscience and perhaps for clinical applications such as restoring degraded memory function; however, existing tools are expensive, cumbersome, and offer limited experimental flexibility. In this article, we propose the Portiloop, a deep learning-based, portable and low-cost closed-loop stimulation system able to target specific brain oscillations. We first document open-hardware implementations that can be constructed from commercially available components. We also provide a fast, lightweight neural network model and an exploration algorithm that automatically optimizes the model hyperparameters to the desired brain oscillation. Finally, we validate the technology on a challenging test case of real-time sleep spindle detection, with results comparable to off-line expert performance on the Massive Online Data Annotation spindle dataset (MODA; group consensus). Software and plans are available to the community as an open science initiative to encourage further development and advance closed-loop neuroscience research [https://github.com/Portiloop].
| S-EPMC9394839 | biostudies-literature
Project description:The prefrontal cortex is extensively involved in social exchange. During dyadic gaze interaction, multiple prefrontal areas exhibit neuronal encoding of social gaze events and context-specific mutual eye contact, supported by a widespread neural mechanism of social gaze monitoring. To explore causal manipulation of real-life gaze interaction, we applied weak closed-loop microstimulations that were precisely triggered by specific social gaze events to three prefrontal areas in monkeys. Microstimulations of orbitofrontal cortex (OFC), but not dorsomedial prefrontal or anterior cingulate cortex, enhanced momentary dynamic social attention in the spatial dimension by decreasing distance of one's gaze fixations relative to partner monkey's eyes. In the temporal dimension, microstimulations of OFC reduced the inter-looking interval for attending to another agent and the latency to reciprocate other's directed gaze. These findings demonstrate that primate OFC serves as a functionally accessible node in controlling dynamic social attention and suggest its potential for a therapeutic brain interface.
| S-EPMC10769221 | biostudies-literature
Project description:Memory failures are frustrating and often the result of ineffective encoding. One approach to improving memory outcomes is through direct modulation of brain activity with electrical stimulation. Previous efforts, however, have reported inconsistent effects when using open-loop stimulation and often target the hippocampus and medial temporal lobes. Here we use a closed-loop system to monitor and decode neural activity from direct brain recordings in humans. We apply targeted stimulation to lateral temporal cortex and report that this stimulation rescues periods of poor memory encoding. This system also improves later recall, revealing that the lateral temporal cortex is a reliable target for memory enhancement. Taken together, our results suggest that such systems may provide a therapeutic approach for treating memory dysfunction.
| S-EPMC5802791 | biostudies-literature
Project description:Pleasurable stimuli, including reward, inhibit pain, but the level of the neuraxis at which they do so and the cerebral processes involved are unknown. Here, we characterized a brain circuitry mediating pain inhibition by reward. Twenty-four healthy participants underwent functional magnetic resonance imaging while playing a wheel of fortune game with simultaneous thermal pain stimuli and monetary wins or losses. As expected, winning decreased pain perception compared to losing. Inter-individual differences in pain modulation by monetary wins relative to losses correlated with activation in the medial orbitofrontal cortex (mOFC). When pain and reward occured simultaneously, mOFCs functional connectivity changed: the signal time course in the mOFC condition-dependent correlated negatively with the signal time courses in the rostral anterior insula, anterior-dorsal cingulate cortex and primary somatosensory cortex, which might signify moment-to-moment down-regulation of these regions by the mOFC. Monetary wins and losses did not change the magnitude of pain-related activation, including in regions that code perceived pain intensity when nociceptive input varies and/or receive direct nociceptive input. Pain inhibition by reward appears to involve brain regions not typically involved in nociceptive intensity coding but likely mediate changes in the significance and/or value of pain.
| S-EPMC5390724 | biostudies-literature
Project description:Optogenetics has been used to orchestrate temporal- and tissue-specific control of neural tissues and offers a wealth of unique advantages for neuromuscular control. Here, we establish a closed-loop functional optogenetic stimulation (CL-FOS) system to control ankle joint position in murine models. Using the measurement of either joint angle or fascicle length as a feedback signal, we compare the controllability of CL-FOS to closed-loop functional electrical stimulation (CL-FES) and demonstrate significantly greater accuracy, lower rise times and lower overshoot percentages. We demonstrate orderly recruitment of motor units and reduced fatigue when performing cyclical movements with CL-FOS compared with CL-FES. We develop and investigate a 3-phase, photo-kinetic model to elucidate the underlying mechanisms for temporal variations in optogenetically activated neuromusculature during closed-loop control experiments. Methods and insights from this study lay the groundwork for the development of closed-loop optogenetic neuromuscular stimulation therapies and devices for peripheral limb control.
| S-EPMC6294002 | biostudies-literature
Project description:Compulsive behaviors (CBs) have been linked to orbitofrontal cortex (OFC) function in animal and human studies. However, brain regions function not in isolation but as components of widely distributed brain networks-such as those indexed via resting-state functional connectivity (RSFC). Sixty-nine individuals with CB disorders were randomized to receive a single session of neuromodulation targeting the left OFC-intermittent theta-burst stimulation (iTBS) or continuous TBS (cTBS)-followed immediately by computer-based behavioral "habit override" training. OFC seeds were used to quantify RSFC following iTBS and following cTBS. Relative to cTBS, iTBS showed increased RSFC between right OFC (Brodmann's area 47) and other areas, including dorsomedial prefrontal cortex (dmPFC), occipital cortex, and a priori dorsal and ventral striatal regions. RSFC connectivity effects were correlated with OFC/frontopolar target engagement and with subjective difficulty during habit-override training. Findings help reveal neural network-level impacts of neuromodulation paired with a specific behavioral context, informing mechanistic intervention development.
| S-EPMC10085574 | biostudies-literature
Project description:The meta-reinforcement learning (meta-RL) framework, which involves RL over multiple timescales, has been successful in training deep RL models that generalize to new environments. It has been hypothesized that the prefrontal cortex may mediate meta-RL in the brain, but the evidence is scarce. Here we show that the orbitofrontal cortex (OFC) mediates meta-RL. We trained mice and deep RL models on a probabilistic reversal learning task across sessions during which they improved their trial-by-trial RL policy through meta-learning. Ca2+/calmodulin-dependent protein kinase II-dependent synaptic plasticity in OFC was necessary for this meta-learning but not for the within-session trial-by-trial RL in experts. After meta-learning, OFC activity robustly encoded value signals, and OFC inactivation impaired the RL behaviors. Longitudinal tracking of OFC activity revealed that meta-learning gradually shapes population value coding to guide the ongoing behavioral policy. Our results indicate that two distinct RL algorithms with distinct neural mechanisms and timescales coexist in OFC to support adaptive decision-making.
| S-EPMC10689244 | biostudies-literature
Project description:Brain stimulation (BStim) encompasses multiple modalities (e.g., deep brain stimulation, responsive neurostimulation) that utilize electrodes implanted in deep brain structures to treat neurological disorders. Currently, BStim is primarily used to treat movement disorders such as Parkinson's, though indications are expanding to include neuropsychiatric disorders like depression and schizophrenia. Traditional BStim systems are "open-loop" and deliver constant electrical stimulation based on manually-determined parameters. Advancements in BStim have enabled development of "closed-loop" systems that analyze neural biomarkers (e.g., local field potentials in the sub-thalamic nucleus) and adjust electrical modulation in a dynamic, patient-specific, and energy efficient manner. These closed-loop systems enable real-time, context-specific stimulation adjustment to reduce symptom burden. Machine learning (ML) has emerged as a vital component in designing these closed-loop systems as ML models can predict / identify presence of disease symptoms based on neural activity and adaptively learn to modulate stimulation. We queried the US National Library of Medicine PubMed database to understand the role of ML in developing closed-loop BStim systems to treat epilepsy, movement disorders, and neuropsychiatric disorders. Both neural and non-neural network ML algorithms have successfully been leveraged to create closed-loop systems that perform comparably to open-loop systems. For disorders in which the underlying neural pathophysiology is relatively well understood (e.g., Parkinson's, essential tremor), most work has involved refining ML models that can classify neural signals as aberrant or normal. The same is seen for epilepsy, where most current research has focused on identifying optimal ML model design and integrating closed-loop systems into existing devices. For neuropsychiatric disorders, where the underlying pathologic neural circuitry is still being investigated, research is focused on identifying biomarkers (e.g., local field potentials from brain nuclei) that ML models can use to identify onset of symptoms and stratify severity of disease.
| S-EPMC10140375 | biostudies-literature
Project description:An optimal choice among alternative behavioral options requires precise anticipatory representations of their possible outcomes. A fundamental question is how such anticipated outcomes are represented in the brain. Reward coding at the level of single cells in the orbitofrontal cortex (OFC) follows a more heterogeneous coding scheme than suggested by studies using functional MRI (fMRI) in humans. Using a combination of multivariate pattern classification and fMRI we show that the reward value of sensory cues can be decoded from distributed fMRI patterns in the OFC. This distributed representation is compatible with previous reports from animal electrophysiology that show that reward is encoded by different neural populations with opposing coding schemes. Importantly, the fMRI patterns representing specific values during anticipation are similar to those that emerge during the receipt of reward. Furthermore, we show that the degree of this coding similarity is related to subjects' ability to use value information to guide behavior. These findings narrow the gap between reward coding in humans and animals and corroborate the notion that value representations in OFC are independent of whether reward is anticipated or actually received.
| S-EPMC2851854 | biostudies-other