Computational mechanisms of curiosity and goal-directed exploration.
Ontology highlight
ABSTRACT:
SUBMITTER: Schwartenbeck P
PROVIDER: S-EPMC6510535 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The function of language in high-order goal-directed human cognition is an important topic at the centre of current debates. Experimental evidence shows that inner speech, representing a self-directed form of language, empowers cognitive processes such as working memory, perception, categorization, and executive functions. Here we study the relations between inner speech and processes like feedback processing and cognitive flexibility. To this aim we propose a computational model that controls an artificial agent who uses inner speech to internally manipulate its representations. The agent is able to reproduce human behavioural data collected during the solution of the Wisconsin Card Sorting test, a neuropsychological test measuring cognitive flexibility, both in the basic condition and when a verbal shadowing protocol is used. The components of the model were systematically lesioned to clarify the specific impact of inner speech on the agent's behaviour. The results indicate that inner speech improves the efficiency of internal representation manipulation. Specifically, it makes the representations linked to specific visual features more disentangled, thus improving the agent's capacity to engage/disengage attention on stimulus features after positive/negative action outcomes. Overall, the model shows how inner speech could improve goal-directed internal manipulation of representations and enhance behavioural flexibility.
| S-EPMC7729881 | biostudies-literature
Project description:Schizophrenia is associated with amotivation and reduced goal-directed behavior, which have been linked to poor functional outcomes. Motivational deficits in schizophrenia are often measured using effort-based decision-making (EBDM) paradigms, revealing consistent alterations in effort expenditure relative to controls. Although these results have generally been interpreted in terms of decreased motivation, the ability to use trial-by-trial changes in reward magnitude or probability of receipt to guide effort allocation may also be affected by cognitive deficits. To date, it remains unclear whether altered performance in EBDM primarily reflects deficits in motivation, cognitive functioning, or both. We applied a newly developed computational modeling approach to the analysis of EBDM data from two previously collected samples comprising 153 patients and 105 controls to determine the extent to which individuals did or did not use available information about reward and probability to guide effort allocation. Half of the participants with schizophrenia failed to incorporate information about reward and probability when making effort-expenditure decisions. The subset of patients who exhibited difficulties using reward and probability information were characterized by greater impairments across measures of cognitive functioning. Interestingly, even within the subset of patients who successfully used reward and probability information to guide effort expenditure, higher levels of negative symptoms related to motivation and avolition were associated with greater effort aversion during the task. Taken together, these data suggest that prior reports of aberrant EBDM in schizophrenia patients are related to both cognitive function and individual differences in negative symptoms. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
| S-EPMC6842449 | biostudies-literature
Project description:What are the cortical neural correlates that distinguish goal-directed and non-goal-directed movements? We investigated this question in the monkey frontal eye field (FEF), which is implicated in voluntary control of saccades. Here, we compared FEF activity associated with goal-directed (G) saccades and non-goal-directed (nG) saccades made by the monkey. Although the FEF neurons discharged before these nG saccades, there were three major differences in the neural activity: First, the variability in spike rate across trials decreased only for G saccades. Second, the local field potential beta-band power decreased during G saccades but did not change during nG saccades. Third, the time from saccade direction selection to the saccade onset was significantly longer for G saccades compared with nG saccades. Overall, our results reveal unexpected differences in neural signatures for G versus nG saccades in a brain area that has been implicated selectively in voluntary control. Taken together, these data add critical constraints to the way we think about saccade generation in the brain.
| S-EPMC8017686 | biostudies-literature
Project description:Curiosity-driven learning is foundational to human cognition. By enabling humans to autonomously decide when and what to learn, curiosity has been argued to be crucial for self-organizing temporally extended learning curricula. However, the mechanisms driving people to set intrinsic goals, when they are free to explore multiple learning activities, are still poorly understood. Computational theories propose different heuristics, including competence measures (e.g., percent correct) and learning progress, that could be used as intrinsic utility functions to efficiently organize exploration. Such intrinsic utilities constitute computationally cheap but smart heuristics to prevent people from laboring in vain on unlearnable activities, while still motivating them to self-challenge on difficult learnable activities. Here, we provide empirical evidence for these ideas by means of a free-choice experimental paradigm and computational modeling. We show that while humans rely on competence information to avoid easy tasks, models that include a learning-progress component provide the best fit to task selection data. These results bridge the research in artificial and biological curiosity, reveal strategies that are used by humans but have not been considered in computational research, and introduce tools for probing how humans become intrinsically motivated to learn and acquire interests and skills on extended time scales.
| S-EPMC8514490 | biostudies-literature
Project description:Experimental and computational studies propose that inner speech boosts categorisation skills and executive functions, making human behaviour more focused and flexible. In addition, many clinical studies highlight a relationship between poor inner-speech and an executive impairment in autism spectrum condition (ASC), but contrasting findings are reported. Here we directly investigate the latter issue through a previously implemented and validated computational model of the Wisconsin Cards Sorting Tests. In particular, the model was applied to explore potential individual differences in cognitive flexibility and inner speech contribution in autistic and neurotypical participants. Our model predicts that the use of inner-speech could increase along the life-span of neurotypical participants but would be reduced in autistic ones. Although we found more attentional failures (i.e., wrong behavioural rule switches) in autistic children/teenagers and more perseverative behaviours in autistic young/older adults, only autistic children and older adults exhibited a lower performance (i.e., fewer consecutive correct rule switches) than matched control groups. Overall, our results corroborate the idea that the reduced use of inner speech could represent a disadvantage for autistic children and autistic older adults. Moreover, the results suggest that cognitive-behavioural therapies should focus on developing inner speech skills in autistic children as this could provide cognitive support throughout their whole life span.
| S-EPMC9392752 | biostudies-literature
Project description:Recent work has given rise to the view that reward-based decision making is governed by two key controllers: a habit system, which stores stimulus-response associations shaped by past reward, and a goal-oriented system that selects actions based on their anticipated outcomes. The current literature provides a rich body of computational theory addressing habit formation, centering on temporal-difference learning mechanisms. Less progress has been made toward formalizing the processes involved in goal-directed decision making. We draw on recent work in cognitive neuroscience, animal conditioning, cognitive and developmental psychology, and machine learning to outline a new theory of goal-directed decision making. Our basic proposal is that the brain, within an identifiable network of cortical and subcortical structures, implements a probabilistic generative model of reward, and that goal-directed decision making is effected through Bayesian inversion of this model. We present a set of simulations implementing the account, which address benchmark behavioral and neuroscientific findings, and give rise to a set of testable predictions. We also discuss the relationship between the proposed framework and other models of decision making, including recent models of perceptual choice, to which our theory bears a direct connection.
| S-EPMC3767755 | biostudies-literature
Project description: Not available
| S-EPMC8724081 | biostudies-literature
Project description:Goal-directed therapy (GDT) can be a vague term, meaning different things to different people and, depending on the clinical environment, sometimes even different things to the same person. It can refer to perioperative fluid management, clinicians driving oxygen delivery to supramaximal values, early treatment of sepsis in the emergency department, and even to restriction of perioperative crystalloids with the goal of maintaining preadmission body weight. Understandably, strong opinions about GDT vary; some clinicians consider it essential for perioperative care, others completely ineffective in critically ill patients. This commentary aims to further position the excellent review by Lees and colleagues in the context of the critical care and perioperative setting.
| S-EPMC2887118 | biostudies-literature
Project description:Finding algorithms that allow agents to discover a wide variety of skills efficiently and autonomously, remains a challenge of Artificial Intelligence. Intrinsically Motivated Goal Exploration Processes (IMGEPs) have been shown to enable real world robots to learn repertoires of policies producing a wide range of diverse effects. They work by enabling agents to autonomously sample goals that they then try to achieve. In practice, this strategy leads to an efficient exploration of complex environments with high-dimensional continuous actions. Until recently, it was necessary to provide the agents with an engineered goal space containing relevant features of the environment. In this article we show that the goal space can be learned using deep representation learning algorithms, effectively reducing the burden of designing goal spaces. Our results pave the way to autonomous learning agents that are able to autonomously build a representation of the world and use this representation to explore the world efficiently. We present experiments in two environments using population-based IMGEPs. The first experiments are performed on a simple, yet challenging, simulated environment. Then, another set of experiments tests the applicability of those principles on a real-world robotic setup, where a 6-joint robotic arm learns to manipulate a ball inside an arena, by choosing goals in a space learned from its past experience.
| S-EPMC7835425 | biostudies-literature
Project description:Several cognitive processes, including instrumental behavior and working memory, are controlled by endocannabinoids acting on cannabinoid receptor 1 (CB1R) in the brain through retrograde and presynaptic inhibition of GABA or glutamate release. However, the temporal mechanisms underlying the control of these cognitive processes by CB1Rs remain largely unknown. Here, we have developed a light-sensitive CB1R chimera (optoCB1R) by replacing the intracellular domains of bovine rhodopsin with those of human CB1R. We demonstrated that light stimulation of optoCB1R triggered canonical CB1R signaling by inhibiting cAMP (but not cGMP or IP1) signaling and activating the MAPK pathway in vitro or in vivo. Moreover, light stimulation of optoCB1R in corticostriatal glutamatergic neurons could temporally inhibit excitatory postsynaptic currents (EPSCs) at the level of seconds. Importantly, transient (3 s) and "time-locked", but not random, activation of optoCB1R signaling in corticostriatal neurons at the time of reward affected animal sensitivity to outcome devaluation and inhibited goal-directed behavior. However, prolonged (~30 min) but not transient (10 or 30 s) activation of astrocytic CB1R signaling in the hippocampus impaired working memory. Consequently, neuronal and astrocytic CB1R signaling differentially regulate working memory and goal-directed behavior through distinct temporal and cellular mechanisms. Ultimately, the pharmacological blockade of adenosine A2AR improved the neuronal and astrocytic CB1R-induced impairments in goal-directed behavior and working memory, possibly through modulation of EPSCs and c-Fos, respectively. Therefore, A2AR may represent a promising target for managing cognitive dysfunction resulting from the use of CB1R drugs.
| S-EPMC10425374 | biostudies-literature