Spatiotemporal dynamics of modality-specific and supramodal word processing.
Ontology highlight
ABSTRACT:
SUBMITTER: Marinkovic K
PROVIDER: S-EPMC3746792 | biostudies-literature | 2003 May
REPOSITORIES: biostudies-literature
Publications
Spatiotemporal dynamics of modality-specific and supramodal word processing.
Marinkovic Ksenija K Dhond Rupali P RP Dale Anders M AM Glessner Maureen M Carr Valerie V Halgren Eric E
Neuron 20030501 3
The ability of written and spoken words to access the same semantic meaning provides a test case for the multimodal convergence of information from sensory to associative areas. Using anatomically constrained magnetoencephalography (aMEG), the present study investigated the stages of word comprehension in real time in the auditory and visual modalities, as subjects participated in a semantic judgment task. Activity spread from the primary sensory areas along the respective ventral processing str ...[more]
Similar Datasets
Project description:Studies with monolingual adults have identified successive stages occurring in different brain regions for processing single written words. We combined magnetoencephalography and magnetic resonance imaging to compare these stages between the first (L1) and second (L2) languages in bilingual adults. L1 words in a size judgment task evoked a typical left-lateralized sequence of activity first in ventral occipitotemporal cortex (VOT: previously associated with visual word-form encoding) and then ventral frontotemporal regions (associated with lexico-semantic processing). Compared to L1, words in L2 activated right VOT more strongly from approximately 135 ms; this activation was attenuated when words became highly familiar with repetition. At approximately 400 ms, L2 responses were generally later than L1, more bilateral, and included the same lateral occipitotemporal areas as were activated by pictures. We propose that acquiring a language involves the recruitment of right hemisphere and posterior visual areas that are not necessary once fluency is achieved.
| S-EPMC2824560 | biostudies-literature
Project description:Attention is inherently biased towards the visual modality during most multisensory scenarios in adults, but the developmental trajectory towards visual dominance has not been fully elucidated. More recent evidence in primates and adult humans suggests a modality-specific stratification of the prefrontal cortex. The current study therefore used functional magnetic resonance imaging (fMRI) to investigate the neuronal correlates of proactive (following cues) and reactive (following probes) cognitive control for simultaneous audio-visual stimulation in 67 healthy adolescents (13-18 years old). Behavioral results were only partially supportive of visual dominance in adolescents, with both reduced response times and accuracy during attend-visual relative to attend-auditory trials. Differential activation of medial and lateral prefrontal cortex for processing incongruent relative to congruent stimuli (reactive control) was also only observed during attend-visual trials. There was no evidence of modality-specific prefrontal cortex stratification during the active processing of multisensory stimuli or during separate functional connectivity analyses. Attention-related modulations were also greater within visual relative to auditory cortex, but were less robust than observed in previous adult studies. Collectively, current results suggest a continued transition towards visual dominance in adolescence, as well as limited modality-specific specialization of prefrontal cortex and attentional modulations of unisensory cortex.
| S-EPMC8379626 | biostudies-literature
Project description:The electrophysiological correlates of conflict processing and cognitive control have been well characterized for the visual modality in paradigms such as the Stroop task. Much less is known about corresponding processes in the auditory modality. Here, electroencephalographic recordings of brain activity were measured during an auditory Stroop task, using three different forms of behavioral response (overt verbal, covert verbal, and manual), that closely paralleled our previous visual Stroop study. As was expected, behavioral responses were slower and less accurate for incongruent than for congruent trials. Neurally, incongruent trials showed an enhanced fronto-central negative polarity wave (N(inc)), similar to the N450 in visual Stroop tasks, with similar variations as a function of behavioral response mode, but peaking ~150 ms earlier, followed by an enhanced positive posterior wave. In addition, sequential behavioral and neural effects were observed that supported the conflict-monitoring and cognitive adjustment hypothesis. Thus, while some aspects of the conflict detection processes, such as timing, may be modality dependent, the general mechanisms would appear to be supramodal.
| S-EPMC3422637 | biostudies-literature
Project description:The cortical processing of auditory-alone, visual-alone, and audiovisual speech information is temporally and spatially distributed, and functional magnetic resonance imaging (fMRI) cannot adequately resolve its temporal dynamics. In order to investigate a hypothesized spatiotemporal organization for audiovisual speech processing circuits, event-related potentials (ERPs) were recorded using electroencephalography (EEG). Stimuli were congruent audiovisual/ba/, incongruent auditory/ba/synchronized with visual/ga/, auditory-only/ba/, and visual-only/ba/and/ga/. Current density reconstructions (CDRs) of the ERP data were computed across the latency interval of 50-250 ms. The CDRs demonstrated complex spatiotemporal activation patterns that differed across stimulus conditions. The hypothesized circuit that was investigated here comprised initial integration of audiovisual speech by the middle superior temporal sulcus (STS), followed by recruitment of the intraparietal sulcus (IPS), followed by activation of Broca's area [Miller, L.M., d'Esposito, M., 2005. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. Journal of Neuroscience 25, 5884-5893]. The importance of spatiotemporally sensitive measures in evaluating processing pathways was demonstrated. Results showed, strikingly, early (<100 ms) and simultaneous activations in areas of the supramarginal and angular gyrus (SMG/AG), the IPS, the inferior frontal gyrus, and the dorsolateral prefrontal cortex. Also, emergent left hemisphere SMG/AG activation, not predicted based on the unisensory stimulus conditions was observed at approximately 160 to 220 ms. The STS was neither the earliest nor most prominent activation site, although it is frequently considered the sine qua non of audiovisual speech integration. As discussed here, the relatively late activity of the SMG/AG solely under audiovisual conditions is a possible candidate audiovisual speech integration response.
| S-EPMC2185744 | biostudies-literature
Project description:Rhythmic joint coordination is ubiquitous in daily-life human activities. In order to coordinate their actions towards shared goals, individuals need to co-regulate their timing and move together at the collective level of behavior. Remarkably, basic forms of coordinated behavior tend to emerge spontaneously as long as two individuals are exposed to each other's rhythmic movements. The present study investigated the dynamics of spontaneous dyadic entrainment, and more specifically how they depend on the sensory modalities mediating informational coupling. By means of a novel interactive paradigm, we showed that dyadic entrainment systematically takes place during a minimalistic rhythmic task despite explicit instructions to ignore the partner. Crucially, the interaction was organized by clear dynamics in a modality-dependent fashion. Our results showed highly consistent coordination patterns in visually-mediated entrainment, whereas we observed more chaotic and more variable profiles in the auditorily-mediated counterpart. The proposed experimental paradigm yields empirical evidence for the overwhelming tendency of dyads to behave as coupled rhythmic units. In the context of our experimental design, it showed that coordination dynamics differ according to availability and nature of perceptual information. Interventions aimed at rehabilitating, teaching or training sensorimotor functions can be ultimately informed and optimized by such fundamental knowledge.
| S-EPMC8443558 | biostudies-literature
Project description:The organization and localization of lexico-semantic information in the brain has been debated for many years. Specifically, lesion and imaging studies have attempted to map the brain areas representing living versus nonliving objects, however, results remain variable. This may be due, in part, to the fact that the univariate statistical mapping analyses used to detect these brain areas are typically insensitive to subtle, but widespread, effects. Decoding techniques, on the other hand, allow for a powerful multivariate analysis of multichannel neural data. In this study, we utilize machine-learning algorithms to first demonstrate that semantic category, as well as individual words, can be decoded from EEG and MEG recordings of subjects performing a language task. Mean accuracies of 76% (chance=50%) and 83% (chance=20%) were obtained for the decoding of living vs. nonliving category or individual words respectively. Furthermore, we utilize this decoding analysis to demonstrate that the representations of words and semantic category are highly distributed both spatially and temporally. In particular, bilateral anterior temporal, bilateral inferior frontal, and left inferior temporal-occipital sensors are most important for discrimination. Successful intersubject and intermodality decoding shows that semantic representations between stimulus modalities and individuals are reasonably consistent. These results suggest that both word and category-specific information are present in extracranially recorded neural activity and that these representations may be more distributed, both spatially and temporally, than previous studies suggest.
| S-EPMC3020243 | biostudies-literature
Project description:Previous research has shown that readers generate mental images of events. Most studies have investigated imagery during the reading of short texts, which also included explicit judgment tasks. In two fMRI studies, we assessed whether modality-specific imagery occurs during naturalistic, discourse comprehension. We identified clauses in the texts that elicited auditory, motor, or visual imagery. In both studies, reading motor imagery clauses was associated with increases in activity in left postcentral and precentral sulci, and reading auditory imagery clauses was associated with increases in left superior temporal gyrus and perisylvian language-related regions. Study 2 compared presentation of connected discourse to a condition in which unconnected sentences were presented, preventing the establishment of global coherence. Sensorimotor imagery was strongest when readers were able to generate a globally coherent discourse representation. Overall, these results suggest that modality-specific imagery occurs during discourse comprehension and it is dependent on the development of discourse-level representations.
| S-EPMC4318524 | biostudies-literature
Project description:Clinical observations showed that schizophrenia (SCZ) patients reported little or no pain under various conditions that are commonly associated with intense painful sensations, leading to a higher risk of morbidity and mortality. However, this phenomenon has received little attention and its underlying neural mechanisms remain unclear. Here, we conducted two experiments combining psychophysics, electroencephalography (EEG), and functional magnetic resonance imaging (fMRI) techniques to investigate neural mechanisms of pain insensitivity in SCZ patients. Specifically, we adopted a stimulus-response paradigm with brief stimuli of different sensory modalities (i.e., nociceptive, non-nociceptive somatosensory, and auditory) to test whether pain insensitivity in SCZ patients is supra-modal or modality-specific, and used EEG and fMRI techniques to clarify its neural mechanisms. We observed that perceived intensities to nociceptive stimuli were significantly smaller in SCZ patients than healthy controls, whereas perceived intensities to non-nociceptive somatosensory and auditory stimuli were not significantly different. The behavioral results were confirmed by stimulus-evoked brain responses sampled by EEG and fMRI techniques, thus verifying the modality-specific nature of the modulation of nociceptive information processing in SCZ patients. Additionally, significant group differences were observed in the spectral power of alpha oscillations in prestimulus EEG and the seed-based functional connectivity in resting-state fMRI (seeds: the thalamus and periaqueductal gray that are key nodes in ascending and descending pain pathways respectively), suggesting a possible contribution of cortical-subcortical dysfunction to the phenomenon. Overall, our study provides insight into the neural mechanisms of pain insensitivity in SCZ and highlights a need for systematic assessments of their pain-related diseases.
| S-EPMC7267942 | biostudies-literature
Project description:Speech comprehension requires the ability to temporally segment the acoustic input for higher-level linguistic analysis. Oscillation-based approaches suggest that low-frequency auditory cortex oscillations track syllable-sized acoustic information and therefore emphasize the relevance of syllabic-level acoustic processing for speech segmentation. How syllabic processing interacts with higher levels of speech processing, beyond segmentation, including the anatomical and neurophysiological characteristics of the networks involved, is debated. In two MEG experiments, we investigate lexical and sublexical word-level processing and the interactions with (acoustic) syllable processing using a frequency-tagging paradigm. Participants listened to disyllabic words presented at a rate of 4 syllables/s. Lexical content (native language), sublexical syllable-to-syllable transitions (foreign language), or mere syllabic information (pseudo-words) were presented. Two conjectures were evaluated: (i) syllable-to-syllable transitions contribute to word-level processing; and (ii) processing of words activates brain areas that interact with acoustic syllable processing. We show that syllable-to-syllable transition information compared to mere syllable information, activated a bilateral superior, middle temporal and inferior frontal network. Lexical content resulted, additionally, in increased neural activity. Evidence for an interaction of word- and acoustic syllable-level processing was inconclusive. Decreases in syllable tracking (cerebroacoustic coherence) in auditory cortex and increases in cross-frequency coupling between right superior and middle temporal and frontal areas were found when lexical content was present compared to all other conditions; however, not when conditions were compared separately. The data provide experimental insight into how subtle and sensitive syllable-to-syllable transition information for word-level processing is.
| S-EPMC10205074 | biostudies-literature
Project description:Learning words is central in human development. However, lacking clear evidence for how or where language is processed in the developing brain, it is unknown whether these processes are similar in infants and adults. Here, we use magnetoencephalography in combination with high-resolution structural magnetic resonance imaging to noninvasively estimate the spatiotemporal distribution of word-selective brain activity in 12- to 18-month-old infants. Infants watched pictures of common objects and listened to words that they understood. A subset of these infants also listened to familiar words compared with sensory control sounds. In both experiments, words evoked a characteristic event-related brain response peaking ?400 ms after word onset, which localized to left frontotemporal cortices. In adults, this activity, termed the N400m, is associated with lexico-semantic encoding. Like adults, we find that the amplitude of the infant N400m is also modulated by semantic priming, being reduced to words preceded by a semantically related picture. These findings suggest that similar left frontotemporal areas are used for encoding lexico-semantic information throughout the life span, from the earliest stages of word learning. Furthermore, this ontogenetic consistency implies that the neurophysiological processes underlying the N400m may be important both for understanding already known words and for learning new words.
| S-EPMC3138516 | biostudies-literature