Dataset Information

A Novel Model for Arbitration Between Planning and Habitual Control Systems.

ABSTRACT: It is well-established that human decision making and instrumental control uses multiple systems, some which use habitual action selection and some which require deliberate planning. Deliberate planning systems use predictions of action-outcomes using an internal model of the agent's environment, while habitual action selection systems learn to automate by repeating previously rewarded actions. Habitual control is computationally efficient but are not very flexible in changing environments. Conversely, deliberate planning may be computationally expensive, but flexible in dynamic environments. This paper proposes a general architecture comprising both control paradigms by introducing an arbitrator that controls which subsystem is used at any time. This system is implemented for a target-reaching task with a simulated two-joint robotic arm that comprises a supervised internal model and deep reinforcement learning. Through permutation of target-reaching conditions, we demonstrate that the proposed is capable of rapidly learning kinematics of the system without a priori knowledge, and is robust to (A) changing environmental reward and kinematics, and (B) occluded vision. The arbitrator model is compared to exclusive deliberate planning with the internal model and exclusive habitual control instances of the model. The results show how such a model can harness the benefits of both systems, using fast decisions in reliable circumstances while optimizing performance in changing environments. In addition, the proposed model learns very fast. Finally, the system which includes internal models is able to reach the target under the visual occlusion, while the pure habitual system is unable to operate sufficiently under such conditions.

SUBMITTER: Sheikhnezhad Fard F

PROVIDER: S-EPMC6637733 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Novel Model for Arbitration Between Planning and Habitual Control Systems.

Sheikhnezhad Fard Farzaneh F Trappenberg Thomas P TP

Frontiers in neurorobotics 20190711

It is well-established that human decision making and instrumental control uses multiple systems, some which use habitual action selection and some which require deliberate planning. Deliberate planning systems use predictions of action-outcomes using an internal model of the agent's environment, while habitual action selection systems learn to automate by repeating previously rewarded actions. Habitual control is computationally efficient but are not very flexible in changing environments. Conv ...[more]

PMID: 31354468

Similar Datasets

Project description:IntroductionHarmful behavior such as smoking may reflect a disturbance in the balance of goal-directed and habitual control. Animal models suggest that habitual control develops after prolonged substance use. In this study, we investigated whether smokers (N = 49) differ from controls (N = 46) in the regulation of goal-directed and habitual behavior. It was also investigated whether individual differences in nicotine dependence levels were associated with habitual responding.MethodsWe used two different multistage instrumental learning tasks that consist of an instrumental learning phase, subsequent outcome devaluation, and a testing phase to measure the balance between goal-directed and habitual responding. The testing phases of these tasks occurred after either appetitive versus avoidance instrumental learning. The appetitive versus aversive instrumental learning stages in the two different tasks modeled positive versus negative reinforcement, respectively.ResultsSmokers and nonsmoking controls did not differ on habitual versus goal-directed control in either task. Individual differences in nicotine dependence within the group of smokers, however, were positively associated with habitual responding after appetitive instrumental learning. This effect seems to be due to impaired stimulus-outcome learning, thereby hampering goal-directed task performance and tipping the balance to habitual responding.ConclusionsThe current finding highlights the importance of individual differences within smokers. For future research, neuroimaging studies are suggested to further unravel the nature of the imbalance between goal-directed versus habitual control in severely dependent smokers by directly measuring activity in the corresponding brain systems.ImplicationsGoal-directed versus habitual behavior in substance use and addiction is highly debated. This study investigated goal-directed versus habitual control in smokers. The findings suggest that smokers do not differ from controls in goal-directed versus habitual control. Individual differences in nicotine dependence within smokers, however, were positively associated with habitual responding after appetitive instrumental learning. This effect seems to be due to impaired stimulus-outcome learning, thereby hampering goal-directed task performance and tipping the balance to habitual responding. These findings add to the ongoing debate on habitual versus goal-directed control in addiction and emphasize the importance of individual differences within smokers.

Dataset Information

A Novel Model for Arbitration Between Planning and Habitual Control Systems.

Publications

A Novel Model for Arbitration Between Planning and Habitual Control Systems.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets