Habitual control of goal selection in humans.
Ontology highlight
ABSTRACT:
SUBMITTER: Cushman F
PROVIDER: S-EPMC4653221 | biostudies-other | 2015 Nov
REPOSITORIES: biostudies-other
Similar Datasets
Project description:IntroductionHarmful behavior such as smoking may reflect a disturbance in the balance of goal-directed and habitual control. Animal models suggest that habitual control develops after prolonged substance use. In this study, we investigated whether smokers (N = 49) differ from controls (N = 46) in the regulation of goal-directed and habitual behavior. It was also investigated whether individual differences in nicotine dependence levels were associated with habitual responding.MethodsWe used two different multistage instrumental learning tasks that consist of an instrumental learning phase, subsequent outcome devaluation, and a testing phase to measure the balance between goal-directed and habitual responding. The testing phases of these tasks occurred after either appetitive versus avoidance instrumental learning. The appetitive versus aversive instrumental learning stages in the two different tasks modeled positive versus negative reinforcement, respectively.ResultsSmokers and nonsmoking controls did not differ on habitual versus goal-directed control in either task. Individual differences in nicotine dependence within the group of smokers, however, were positively associated with habitual responding after appetitive instrumental learning. This effect seems to be due to impaired stimulus-outcome learning, thereby hampering goal-directed task performance and tipping the balance to habitual responding.ConclusionsThe current finding highlights the importance of individual differences within smokers. For future research, neuroimaging studies are suggested to further unravel the nature of the imbalance between goal-directed versus habitual control in severely dependent smokers by directly measuring activity in the corresponding brain systems.ImplicationsGoal-directed versus habitual behavior in substance use and addiction is highly debated. This study investigated goal-directed versus habitual control in smokers. The findings suggest that smokers do not differ from controls in goal-directed versus habitual control. Individual differences in nicotine dependence within smokers, however, were positively associated with habitual responding after appetitive instrumental learning. This effect seems to be due to impaired stimulus-outcome learning, thereby hampering goal-directed task performance and tipping the balance to habitual responding. These findings add to the ongoing debate on habitual versus goal-directed control in addiction and emphasize the importance of individual differences within smokers.
| S-EPMC7004226 | biostudies-literature
Project description:Our daily decisions are governed by the arbitration between goal-directed and habitual strategies. However, the neurochemical basis of this arbitration is unclear. We assessed the contribution of dopaminergic, serotonergic, and opioidergic systems to this balance across reward and loss domains. Thirty-nine participants (17 healthy controls, 15 patients with pathological gambling, and 7 with binge eating disorder) underwent positron emission tomography (PET) imaging with [18F]FDOPA, [11C]MADAM and [11C]carfentanil to assess presynaptic dopamine, and serotonin transporter and mu-opioid receptor binding potential. Separately, participants completed a modified two-step task, which quantifies the degree to which decision-making is influenced by goal-directed or habitual strategies. All participants completed a version with reward outcomes; healthy controls additionally completed a version with loss outcomes. In the context of rewarding outcomes, we found that greater serotonin transporter binding potential in prefrontal regions was associated with habitual control, while greater serotonin transporter binding potential in the putamen was marginally associated with goal-directed control; however, the findings were no longer significant when controlling for the opposing valence (loss). In blocks with loss outcomes, we found that the opioidergic system, specifically greater [11C]carfentanil binding potential, was positively associated with goal-directed control and negatively associated with habit-directed control. Our findings illuminate the complex neurochemical basis of goal-directed and habitual behavior, implicating differential roles for prefrontal and subcortical serotonin in decision-making across healthy and pathological populations.
| S-EPMC7054261 | biostudies-literature
Project description:Decision-making in the real world presents the challenge of requiring flexible yet prompt behavior, a balance that has been characterized in terms of a trade-off between a slower, prospective goal-directed model-based (MB) strategy and a fast, retrospective habitual model-free (MF) strategy. Theory predicts that flexibility to changes in both reward values and transition contingencies can determine the relative influence of the two systems in reinforcement learning, but few studies have manipulated the latter. Therefore, we developed a novel two-level contingency change task in which transition contingencies between states change every few trials; MB and MF control predict different responses following these contingency changes, allowing their relative influence to be inferred. Additionally, we manipulated the rate of contingency changes in order to determine whether contingency change volatility would play a role in shifting subjects between a MB and MF strategy. We found that human subjects employed a hybrid MB/MF strategy on the task, corroborating the parallel contribution of MB and MF systems in reinforcement learning. Further, subjects did not remain at one level of MB/MF behaviour but rather displayed a shift towards more MB behavior over the first two blocks that was not attributable to the rate of contingency changes but rather to the extent of training. We demonstrate that flexibility to contingency changes can distinguish MB and MF strategies, with human subjects utilizing a hybrid strategy that shifts towards more MB behavior over blocks, consequently corresponding to a higher payoff.
| S-EPMC5634647 | biostudies-literature
Project description:Do people differ in their propensity to form habits? The current study related individual differences in habitual performance on the slips-of-action task to habit formation in real life. To this end, we developed a novel key-cover procedure that controls for the amount of repetition and motivation within a naturalistic setting. Participants received a key cover for the key to their home, which after several weeks was switched with a key cover that was previously attached to a dummy key. Participants recorded effort, time, attention, and mistakes in the key-selection process. Results were in line with established properties of habits, as attention decreased in the learning phase, yet effort, time, and mistakes increased after the key-cover switch. Performance on the slips-of-action task correlated negatively with changes in attention in the real-life key-cover task. This negative correlation may reflect that flexible behavioral adjustment requires more attention in people with a relatively weak goal-directed system.
| S-EPMC6105188 | biostudies-literature
Project description:According to dual-system accounts, instrumental learning is supported by both a goal-directed and a habitual system. Although behavioral control by the goal-directed system, through outcome-action associations, dominates with moderate training, stimulus-response associations are thought to form concurrently in the habit system. It is therefore challenging to isolate the neural substrate of the goal-directed system in neuroimaging research with healthy human volunteers. Recently, however, de Wit et al. (2007) developed an instrumental discrimination task that distinguishes between goal-directed and habit-based responding. In this task, cues are congruent, unrelated, or incongruent with subsequent outcomes. Whereas performance on congruent and control trials can be supported by both the goal-directed and habitual system, performance on the incongruent discrimination relies solely on the habit system. In the present study, we used this task with healthy participants undergoing functional magnetic resonance imaging to demonstrate that engagement of the goal-directed system during learning is reflected in increased activity in the ventromedial prefrontal cortex. Moreover, using a subsequent outcome devaluation manipulation, we show that this area is involved in guiding decision making when goal values change, even in the absence of external cues to guide performance. We can therefore exclude a purely Pavlovian account of ventromedial prefrontal function and unequivocally demonstrate its involvement in the acquisition as well as deployment of goal-directed knowledge.
| S-EPMC3443853 | biostudies-literature
Project description:Instrumental learning is mediated by goal-directed and habit systems in the brain. While rodent studies implicate distinct prefrontal/striatal regions in goal-directed and habit learning, neural systems underpinning these two processes in humans remain poorly understood. Here, using a validated discrimination learning task that distinguishes goal-directed learning from habit learning in 72 subjects in fMRI, we investigated the corticostriatal correlates of goal-directed learning and tested whether brain activation during learning is associated with trait motivation and behavioral performance in the post-learning test phase. Participants showed enhanced activation in medial prefrontal and posterior cingulate cortices during goal-directed action selection in the training phase, whereas habitual action selection activated bilateral insula, bilateral dorsal caudate and left precentral gyrus. In addition, early phase of learning was associated with increased activation in the frontoparietal control network and dorsal striatum, whereas default mode regions depicted increased activation in the late phase. Finally, avoidance motivation scores measured by Behavioral Inhibition/Activation System (BIS/BAS) correlated with accuracy during goal-directed learning and showed a nominally significant correlation with activation in dorsomedial prefrontal cortex during goal-directed acquisition of stimuli. These findings reveal the temporal dynamics of instrumental behavior and suggest that avoidance motivation predicts performance and brain activity during goal-directed learning.
| S-EPMC5519538 | biostudies-literature
Project description:Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other. We found, based on subjects' choices and reaction times, that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions. Furthermore, based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects' behavior than a family of flat models. Although these findings do not rule out all possible model-free accounts of instrumental conditioning, they do show such accounts are not necessary to explain habitual actions and provide a new basis for understanding how goal-directed and habitual action control interact.
| S-EPMC3854489 | biostudies-literature
Project description:Successful adaptation to complex environments depends on the balance of at least two systems: a flexible but slow goal-directed system encoding action-outcome associations and an efficient but rigid habitual system linking responses to preceding stimuli. Recent evidence suggests that the inferolateral prefrontal cortex (ilPFC), a region well known to contribute to cognitive control processes, may play a crucial role in the balance of goal-directed and habitual responding. This evidence, however, comes mainly from correlational data and whether the ilPFC is indeed causally involved in the goal-directed vs. habitual control of behavior is unclear. Here, we used neuro-navigated theta-burst stimulation (TBS) to either inhibit or enhance right ilPFC functionality before participants completed an instrumental learning task designed to probe goal-directed vs. habitual behavioral control. TBS did not affect overall learning performance. However, participants that had received inhibitory TBS were less able to adapt their behavior to altered task demands, indicating a shift from goal-directed towards more habitual control of behavior. Sham or excitatory TMS groups showed no such effect and were comparable in their performance to an unstimulated control group. Our findings indicate a causal role of the ilPFC in the balance of goal-directed vs. habitual control of behavior.
| S-EPMC6010441 | biostudies-literature
Project description:A brain-computer interface (BCI) can be used to accomplish a task without requiring motor output. Two major control strategies used by BCIs during task completion are process control and goal selection. In process control, the user exerts continuous control and independently executes the given task. In goal selection, the user communicates their goal to the BCI and then receives assistance executing the task. A previous study has shown that goal selection is more accurate and faster in use. An unanswered question is, which control strategy is easier to learn? This study directly compares goal selection and process control while learning to use a sensorimotor rhythm-based BCI. Twenty young healthy human subjects were randomly assigned either to a goal selection or a process control-based paradigm for eight sessions. At the end of the study, the best user from each paradigm completed two additional sessions using all paradigms randomly mixed. The results of this study were that goal selection required a shorter training period for increased speed, accuracy, and information transfer over process control. These results held for the best subjects as well as in the general subject population. The demonstrated characteristics of goal selection make it a promising option to increase the utility of BCIs intended for both disabled and able-bodied users.
| S-EPMC3279116 | biostudies-literature
Project description:Everyday decision-making is supported by a dual-system of control comprised of parallel goal-directed and habitual systems. Over the past decade, the two-stage Markov decision task has become popularized for its ability to dissociate between goal-directed and habitual decision-making. While a handful of studies have implemented decision-making tasks online, only one study has validated the task by comparing in-person and web-based performance on the two-stage task in children and young adults. To date, no study has validated the dissociation of goal-directed and habitual behaviors in older adults online. Here, we implemented and validated a web-based version of the two-stage Markov task using parameter simulation and recovery and compared behavioral results from online and in-person participation on the two-stage task in both young and healthy older adults. We found no differences in estimated free parameters between online and in-person participation on the two-stage task. Further, we replicate previous findings that young adults are more goal-directed than older adults both in-person and online. Overall, this work demonstrates that the implementation and use of the two-stage Markov decision task for remote participation is feasible in the older adult demographic, which would allow for the study of decision-making with larger and more diverse samples.
| S-EPMC8276057 | biostudies-literature