Multiple spatial frames for immersive working memory.
Ontology highlight
ABSTRACT: As we move around, relevant information that disappears from sight can still be held in working memory to serve upcoming behaviour. How we maintain and select visual information as we move through the environment remains poorly understood because most laboratory tasks of working memory rely on removing visual material while participants remain still. We used virtual reality to study visual working memory following self-movement in immersive environments. Directional biases in gaze revealed the recruitment of more than one spatial frame for maintaining and selecting memoranda following self-movement. The findings bring the important realization that multiple spatial frames support working memory in natural behaviour. The results also illustrate how virtual reality can be a critical experimental tool to characterize this core memory system.
Project description:Prior results on the spatial integration of layouts within a room differed regarding the reference frame that participants used for integration. We asked whether these differences also occur when integrating 2D screen views and, if so, what the reasons for this might be. In four experiments we showed that integrating reference frames varied as a function of task familiarity combined with processing time, cues for spatial transformation, and information about action requirements paralleling results in the 3D case. Participants saw part of an object layout in screen 1, another part in screen 2, and reacted on the integrated layout in screen 3. Layout presentations between two screens coincided or differed in orientation. Aligning misaligned screens for integration is known to increase errors/latencies. The error/latency pattern was thus indicative of the reference frame used for integration. We showed that task familiarity combined with self-paced learning, visual updating, and knowing from where to act prioritized the integration within the reference frame of the initial presentation, which was updated later, and from where participants acted respectively. Participants also heavily relied on layout intrinsic frames. The results show how humans flexibly adjust their integration strategy to a wide variety of conditions.
Project description:The contexts for action may be only transiently visible, accessible, and relevant. The corticobasal ganglia (BG) circuit addresses these demands by allowing the right motor plans to drive action at the right times, via a BG-mediated gate on motor representations. A long-standing hypothesis posits these same circuits are replicated in more rostral brain regions to support gating of cognitive representations. Key evidence now supports the prediction that BG can act as a gate on the input to working memory, as a gate on its output, and as a means of reallocating working memory representations rendered irrelevant by recent events. These discoveries validate key tenets of many computational models, circumscribe motor and cognitive models of recurrent cortical dynamics alone, and identify novel directions for research on the mechanisms of higher-level cognition.
Project description:Many researchers have noted that the functional architecture of the human brain is relatively invariant during task performance and the resting state. Indeed, intrinsic connectivity networks (ICNs) revealed by resting-state functional connectivity analyses are spatially similar to regions activated during cognitive tasks. This suggests that patterns of task-related activation in individual subjects may result from the engagement of one or more of these ICNs; however, this has not been tested. We used a novel analysis, spatial multiple regression, to test whether the patterns of activation during an N-back working memory task could be well described by a linear combination of ICNs delineated using Independent Components Analysis at rest. We found that across subjects, the cingulo-opercular Set Maintenance ICN, as well as right and left Frontoparietal Control ICNs, were reliably activated during working memory, while Default Mode and Visual ICNs were reliably deactivated. Further, involvement of Set Maintenance, Frontoparietal Control, and Dorsal Attention ICNs was sensitive to varying working memory load. Finally, the degree of left Frontoparietal Control network activation predicted response speed, while activation in both left Frontoparietal Control and Dorsal Attention networks predicted task accuracy. These results suggest that a close relationship between resting-state networks and task-evoked activation is functionally relevant for behavior, and that spatial multiple regression analysis is a suitable method for revealing that relationship.
Project description:A common assumption about spatial memory is that it is organized along one or more reference directions such that access to memory is easier along directions aligned with the reference direction(s). This assumption rests to no small part on frequently replicated alignment effects arising in judgment of relative direction. In this contribution, we report an experiment designed to investigate a possible alternative explanation of alignment effects. By contrasting performance in a judgment of relative direction task with performance in an ego perspective taking task, we tested to what extent alignment effects arise from encoding of relations in addition to or instead of from organization along reference directions. Experimental results suggest little if any contribution of relation encoding on alignment effects, thus lending further support to the assumption of reference directions in spatial memory. Data from both tasks yielded the same alignment effects and provided evidence for a single direction being encoded in memory. Moreover, our results shed new light on and raise questions concerning differential sensorimotor and cognitive influence on spatial memory use. While both influence memory use, systematic bias seems to arise solely from reference directions, along which memory is organized.
Project description:Extended maintenance delays decrease the accuracy of information stored in spatial working memory. In order to elucidate the network underlying sustained spatial working memory, 16 subjects were scanned using fast event-related fMRI as they performed an oculomotor delayed response task containing trials with "short" (2.5 s) or "long" (10 s) delay periods. Multiple cortical and subcortical regions were common to both delay trial types indicating core task regions. Three patterns of activity were found in a subset of core regions that reflect underlying processes: maintenance-related (e.g., left FEF, right supramarginal gyrus (SMG)), response planning-related (e.g., right FEF, SEF), and motor response-related (e.g., lateral cerebellum (declive)) activation. Several regions were more active during long than short delay trials, including multiple sites in DLPFC (BA 9, 46), indicating a circuitry dynamically recruited to support sustained working memory. Our results suggest that specialized brain processes support extended periods of working memory.
Project description:Each time we make an eye movement, positions of objects on the retina change. In order to keep track of relevant objects their positions have to be updated. The situation becomes even more complex if the object is no longer present in the world and has to be held in memory. In the present study, we used saccadic curvature to investigate the time-course of updating a memorized location across saccades. Previous studies have shown that a memorized location competes with a saccade target for selection on the oculomotor map, which leads to saccades curving away from it. In our study participants performed a sequence of two saccades while keeping a location in memory. The trajectory of the second saccade was used to measure when the memorized location was updated after the first saccade. The results showed that the memorized location was rapidly updated with the eyes curving away from its spatial coordinates within 130 ms after the first eye movement. The time-course of updating was comparable to the updating of an exogenously attended location, and depended on how well the location was memorized.
Project description:Ample research in visual working memory (VWM) has demonstrated that the memorized items are maintained in integrated spatial configurations, even when the spatial context is task irrelevant. These insights were obtained in studies in which participants were provided with the information they memorized. However, the encoding of provided information is only one aspect of memory. In everyday life, individuals often construct their own memory representations, an aspect of memory we have previously termed self-initiated (SI) working memory. In this study, we employed a SI VWM task in which participants selected the visual targets they memorized. The spatial locations of the targets were task irrelevant. Nevertheless, we were interested to see whether participants would construct spatially structured memory representations, which would suggest that they intended to maintain the visual targets as integrated spatial configurations. The results of two experiments demonstrated that participants constructed spatially structured configurations relative to random displays. Specifically, participants selected visual targets in close spatial proximity and constructed spatial sequences with short distances and fewer path crossings. When asked to construct configurations for a hypothetical competitor in a memory contest, participants disrupted the spatial structure by selecting visual targets that were further apart and by increasing the distances between them, which suggests that these characteristics were under their control. At the end of each experiment, participants provided verbal descriptions of the strategies they used to construct the memory displays. While the spatial structure of the SI memory representations was robust, it was absent from the participants' explicit descriptions, which focused on non-spatial strategies. Participants reported selecting items based, most frequently, on semantic categories and visual features. Taken together, these results demonstrated that participants had access to the metacognitive knowledge on the spatial structure of VWM representations, knowledge they manipulated to construct memory representations that enhanced or disrupted memory performance. While having a profound impact on behavior, this metacognitive knowledge on spatial structure remained implicit, as it was absent from the participants' verbal reports. Viewed from a larger perspective, this study explores how individuals interact with the world by actively structuring their surroundings to maximize cognitive performance.
Project description:In typical visual working memory tasks, participants report the color of a previously studied item at some probed location. Alternatively, in some recent studies, a color is probed and participants must report the item's location. There is a surprising difference between these tasks: in location reports participants almost never guess randomly as they do when reporting color, but often incorrectly report the locations of non-probed items. This finding has been taken as evidence for feature binding errors in memory, and evidence against discrete capacity models, which predict that pure guessing should occur. We test an alternative possibility: that non-target responses are guesses, but intelligent ones. In particular, when asked to report the location of an item for which participants have no memory, they may guess near locations where they know something was presented. Here we present false-probe trials in which a color is probed that was not actually studied, and find that the responses, which are necessarily guesses, are nonetheless centered around studied locations. Moreover, we find that the confidence ratings for non-target responses are low, and similar to confidence for uniformly distributed guesses. In a second experiment, we find that manipulating the retention interval, which is known to affect guess rates, changes the rate of these low-confidence non-target responses. These results suggest that the tendency to report locations of non-probed items reflects a good guessing strategy; not something fundamental about how features and objects are represented in working memory.
Project description:In dynamic multisensory environments, the perceptual system corrects for discrepancies arising between modalities. For instance, in the ventriloquism aftereffect (VAE), spatial disparities introduced between visual and auditory stimuli lead to a perceptual recalibration of auditory space. Previous research has shown that the VAE is underpinned by multiple recalibration mechanisms tuned to different timescales, however it remains unclear whether these mechanisms use common or distinct spatial reference frames. Here we asked whether the VAE operates in eye- or head-centred reference frames across a range of adaptation timescales, from a few seconds to a few minutes. We developed a novel paradigm for selectively manipulating the contribution of eye- versus head-centred visual signals to the VAE by manipulating auditory locations relative to either the head orientation or the point of fixation. Consistent with previous research, we found both eye- and head-centred frames contributed to the VAE across all timescales. However, we found no evidence for an interaction between spatial reference frames and adaptation duration. Our results indicate that the VAE is underpinned by multiple spatial reference frames that are similarly leveraged by the underlying time-sensitive mechanisms.
Project description:Working memory (WM) enables the maintenance and manipulation of information relevant to behavioral goals. Variability in WM ability is strongly correlated with IQ [1], and WM function is impaired in many neurological and psychiatric disorders [2, 3], suggesting that this system is a core component of higher cognition. WM storage is thought to be mediated by patterns of activity in neural populations selective for specific properties (e.g., color, orientation, location, and motion direction) of memoranda [4-13]. Accordingly, many models propose that differences in the amplitude of these population responses should be related to differences in memory performance [14, 15]. Here, we used functional magnetic resonance imaging and an image reconstruction technique based on a spatial encoding model [16] to visualize and quantify population-level memory representations supported by multivoxel patterns of activation within regions of occipital, parietal and frontal cortex while participants precisely remembered the location(s) of zero, one, or two small stimuli. We successfully reconstructed images containing representations of the remembered-but not forgotten-locations within regions of occipital, parietal, and frontal cortex using delay-period activation patterns. Critically, the amplitude of representations of remembered locations and behavioral performance both decreased with increasing memory load. These results suggest that differences in visual WM performance between memory load conditions are mediated by changes in the fidelity of large-scale population response profiles distributed across multiple areas of human cortex.