Influences on the Test-Retest Reliability of Functional Connectivity MRI and its Relationship with Behavioral Utility.
Ontology highlight
ABSTRACT:
SUBMITTER: Noble S
PROVIDER: S-EPMC6248395 | biostudies-literature | 2017 Nov
REPOSITORIES: biostudies-literature
Publications
Influences on the Test-Retest Reliability of Functional Connectivity MRI and its Relationship with Behavioral Utility.
Noble Stephanie S Spann Marisa N MN Tokoglu Fuyuze F Shen Xilin X Constable R Todd RT Scheinost Dustin D
Cerebral cortex (New York, N.Y. : 1991) 20171101 11
Best practices are currently being developed for the acquisition and processing of resting-state magnetic resonance imaging data used to estimate brain functional organization-or "functional connectivity." Standards have been proposed based on test-retest reliability, but open questions remain. These include how amount of data per subject influences whole-brain reliability, the influence of increasing runs versus sessions, the spatial distribution of reliability, the reliability of multivariate ...[more]
Similar Datasets
Project description:Resting functional MRI studies of the infant brain are increasingly becoming an important tool in developmental neuroscience. Whereas the test-retest reliability of functional connectivity (FC) measures derived from resting fMRI data have been characterized in the adult and child brain, similar assessments have not been conducted in infants. In this study, we examined the intra-session test-retest reliability of FC measures from 119 infant brain MRI scans from four neurodevelopmental studies. We investigated edge-level and subject-level reliability within one MRI session (between and within runs) measured by the Intraclass correlation coefficient (ICC). First, using an atlas-based approach, we examined whole-brain connectivity as well as connectivity within two common resting fMRI networks - the default mode network (DMN) and the sensorimotor network (SMN). Second, we examined the influence of run duration, study site, and scanning manufacturer (e.g., Philips and General Electric) on ICCs. Lastly, we tested spatial similarity using the Jaccard Index from networks derived from independent component analysis (ICA). Consistent with resting fMRI studies from adults, our findings indicated poor edge-level reliability (ICC = 0.14-0.18), but moderate-to-good subject-level intra-session reliability for whole-brain, DMN, and SMN connectivity (ICC = 0.40-0.78). We also found significant effects of run duration, site, and scanning manufacturer on reliability estimates. Some ICA-derived networks showed strong spatial reproducibility (e.g., DMN, SMN, and Visual Network), and were labelled based on their spatial similarity to analogous networks measured in adults. These networks were reproducibly found across different study sites. However, other ICA-networks (e.g. Executive Control Network) did not show strong spatial reproducibility, suggesting that the reliability and/or maturational course of functional connectivity may vary by network. In sum, our findings suggest that developmental scientists may be on safe ground examining the functional organization of some major neural networks (e.g. DMN and SMN), but judicious interpretation of functional connectivity is essential to its ongoing success.
| S-EPMC8335644 | biostudies-literature
Project description:The reliability of magnetoencephalography (MEG) resting-state functional connectivity in schizophrenia (SZ) is unknown as previous research has focused on healthy controls (HC). Here, we examined reliability in 26 participants (13-SZ, 13-HC). Eyes opened and eyes closed resting-state data were collected on 4 separate occasions during 2 visits, 1 week apart. For source modeling, we used minimum norm software to apply dynamic statistical parametric mapping. Source analyses compared the following functional connectivity metrics from each data run: coherence (coh), imaginary coherence (imcoh), pairwise phase consistency (ppc), phase-locking value (plv), phase lag index (pli), weighted phase lag index (wpli), and weighted phase lag index debiased (wpli2). Intraclass correlation coefficients (ICCs) were calculated for whole brain, network, and network pair averages. For reliability, ICCs above 0.75 = excellent, above 0.60 = good, above 0.40 = fair, and below 0.40 = poor reliability. We found the reliability of these metrics varied greatly depending on frequency band, network, network pair, and participant group examined. Broadband (1-58 Hz) whole brain averages in both HC and SZ showed excellent reliability for wpli2, and good to fair reliability for ppc, plv, and coh. Broadband network averages showed excellent to good reliability across 1 hour and 1 week for coh, imcoh, ppc, plv, wpli within default mode, cognitive control, and visual networks in HC, while the same metrics had excellent to fair reliability in SZ. Regional network pair averages showed good to fair reliability for coh, ppc, plv within default mode, cognitive control and visual network pairs in HC and SZ. In general, HC had higher reliability compared to SZ, and the default mode, cognitive control, and visual networks had higher reliability compared to somatosensory and auditory networks. Similar reliability levels occurred for both eyes opened and eyes closed resting-states for most metrics. The functional connectivity metrics of coh, ppc, and plv performed best across 1 hour and 1 week in HC and SZ. We also found that SZ had reduced coh, plv, and ppc in the dmn average and pair values indicating dysconnectivity in SZ. These findings encourage collecting both eyes opened and eyes closed resting-state MEG, while demonstrating that clinical populations may differ in reliability.
| S-EPMC7772354 | biostudies-literature
Project description:Functional connectivity analysis has become a powerful tool for probing the human brain function and its breakdown in neuropsychiatry disorders. So far, most studies adopted resting-state paradigm to examine functional connectivity networks in the brain, thanks to its low demand and high tolerance that are essential for clinical studies. However, the test-retest reliability of resting-state connectivity measures is moderate, potentially due to its low behavioral constraint. On the other hand, naturalistic neuroimaging paradigms, an emerging approach for cognitive neuroscience with high ecological validity, could potentially improve the reliability of functional connectivity measures. To test this hypothesis, we characterized the test-retest reliability of functional connectivity measures during a natural viewing condition, and benchmarked it against resting-state connectivity measures acquired within the same functional magnetic resonance imaging (fMRI) session. We found that the reliability of connectivity and graph theoretical measures of brain networks is significantly improved during natural viewing conditions over resting-state conditions, with an average increase of almost 50% across various connectivity measures. Not only sensory networks for audio-visual processing become more reliable, higher order brain networks, such as default mode and attention networks, but also appear to show higher reliability during natural viewing. Our results support the use of natural viewing paradigms in estimating functional connectivity of brain networks, and have important implications for clinical application of fMRI. Hum Brain Mapp 38:2226-2241, 2017. © 2017 Wiley Periodicals, Inc.
| S-EPMC6867176 | biostudies-literature
Project description:Functional connectivity (FC) has become a leading method for resting-state functional magnetic resonance imaging (rs-fMRI) analysis. However, the majority of the previous studies utilized pairwise, temporal synchronization-based FC. Recently, high-order FC (HOFC) methods were proposed with the idea of computing "correlation of correlations" to capture high-level, more complex associations among the brain regions. There are two types of HOFC. The first type is topographical profile similarity-based HOFC (tHOFC) and its variant, associated HOFC (aHOFC), for capturing different levels of HOFC. Instead of measuring the similarity of the original rs-fMRI signals with the traditional FC (low-order FC, or LOFC), tHOFC measures the similarity of LOFC profiles (i.e., a set of LOFC values between a region and all other regions) between each pair of brain regions. The second type is dynamics-based HOFC (dHOFC) which defines the quadruple relationship among every four brain regions by first calculating two pairwise dynamic LOFC "time series" and then measuring their temporal synchronization (i.e., temporal correlation of the LOFC fluctuations, not the BOLD fluctuations). Applications have shown the superiority of HOFC in both disease biomarker detection and individualized diagnosis than LOFC. However, no study has been carried out for the assessment of test-retest reliability of different HOFC metrics. In this paper, we systematically evaluate the reliability of the two types of HOFC methods using test-retest rs-fMRI data from 25 (12 females, age 24.48 ± 2.55 years) young healthy adults with seven repeated scans (with interval = 3-8 days). We found that all HOFC metrics have satisfactory reliability, specifically (1) fair-to-good for tHOFC and aHOFC, and (2) fair-to-moderate for dHOFC with relatively strong connectivity strength. We further give an in-depth analysis of the biological meanings of each HOFC metric and highlight their differences compared to the LOFC from the aspects of cross-level information exchanges, within-/between-network connectivity, and modulatory connectivity. In addition, how the dynamic analysis parameter (i.e., sliding window length) affects dHOFC reliability is also investigated. Our study reveals unique functional associations characterized by the HOFC metrics. Guidance and recommendations for future applications and clinical research using HOFC are provided. This study has made a further step toward unveiling more complex human brain connectome.
| S-EPMC5539178 | biostudies-other
Project description:Resting-state functional MRI (rs-fMRI) has emerged as a powerful tool for investigating brain functional connectivity (FC). Research in recent years has focused on assessing the reliability of FC across younger subjects within and between scan-sessions. Test-retest reliability in resting-state functional connectivity (RSFC) has not yet been examined in older adults. In this study, we investigated age-related differences in reliability and stability of RSFC across scans. In addition, we examined how global signal regression (GSR) affects RSFC reliability and stability. Three separate resting-state scans from 29 younger adults (18-35 yrs) and 26 older adults (55-85 yrs) were obtained from the International Consortium for Brain Mapping (ICBM) dataset made publically available as part of the 1000 Functional Connectomes project www.nitrc.org/projects/fcon_1000. 92 regions of interest (ROIs) with 5 cubic mm radius, derived from the default, cingulo-opercular, fronto-parietal and sensorimotor networks, were previously defined based on a recent study. Mean time series were extracted from each of the 92 ROIs from each scan and three matrices of z-transformed correlation coefficients were created for each subject, which were then used for evaluation of multi-scan reliability and stability. The young group showed higher reliability of RSFC than the old group with GSR (p-value = 0.028) and without GSR (p-value <0.001). Both groups showed a high degree of multi-scan stability of RSFC and no significant differences were found between groups. By comparing the test-retest reliability of RSFC with and without GSR across scans, we found significantly higher proportion of reliable connections in both groups without GSR, but decreased stability. Our results suggest that aging is associated with reduced reliability of RSFC which itself is highly stable within-subject across scans for both groups, and that GSR reduces the overall reliability but increases the stability in both age groups and could potentially alter group differences of RSFC.
| S-EPMC3515585 | biostudies-literature
Project description:BackgroundFunctional MRI (fMRI) tasks are increasingly being used to advance knowledge of the etiology and maintenance of obesity and eating disorders. Thus, understanding the test-retest reliability of BOLD signal contrasts from these tasks is important.ObjectivesTo evaluate test-retest reliability of responses in reward-related brain regions to food receipt paradigms (palatable tastes, anticipated palatable tastes), food picture paradigms (high-calorie food pictures), a monetary reward paradigm (winning money and anticipating winning money), and a thin female model picture paradigm (thin female model pictures).MethodWe conducted secondary univariate contrast-based analyses in data drawn from 4 repeated-measures fMRI studies. Participants (Study 1: N = 60, mean [M] age = 15.2 ± 1.1 y; Study 2: N = 109, M age = 15.1 ± 0.9 y; Study 3: N = 39, M age = 21.2 ± 3.7 y; Study 4: N = 62, M age = 29.7 ± 6.2 y) completed the same tasks over 3-wk to 3-y test-retest intervals. Studies 3 and 4 included participants with eating disorders and obesity, respectively.ResultsTest-retest reliability of the food receipt and food picture paradigms was poor, with average ICC values ranging from 0.07 to 0.20. The monetary reward paradigm and the thin female model picture paradigm also showed poor test-retest reliability: average ICC values 0.21 and 0.12, respectively. Although several regions demonstrated moderate to good test-retest reliability, these results did not replicate across studies using similar paradigms. In Studies 3 and 4, but not Study 1, test-retest reliability in visual processing regions was moderate to good when contrasting single conditions with a low-level baseline.ConclusionsResults underscore the importance of examining the temporal reliability of fMRI tasks and call for the development and use of well-validated standardized fMRI tasks in eating- and obesity-related studies that can provide more reliable measures of neural activation. The trials were registered at clinicaltrials.gov as NCT02084836, NCT01949636, NCT03261050, and NCT03375853.
| S-EPMC8326039 | biostudies-literature
Project description:IntroductionFunctional MRI (fMRI) is commonly used to investigate the neural mechanisms underlying psychological processes and behavioral responses. However, to draw well-founded conclusions from fMRI studies, more research on the reliability of fMRI is needed.MethodsWe invited a sample of 41 female students to participate in two identical fMRI sessions, separated by 5 weeks on average. To investigate the potential effect of left-handedness on the stability of neural activity, we oversampled left-handed participants (N = 20). Inside the scanner, we presented photographs of familiar and unfamiliar children's faces preceded by neutral and threatening primes to the participants. We calculated intraclass correlations (ICCs) to investigate the test-retest reliability of peak activity in areas that showed significant activity during the first session (primary visual cortex, fusiform face area, inferior frontal gyrus, and superior temporal gyrus). In addition, we examined how many trials were needed to reliably measure the effects.ResultsAcross all participants, only fusiform face area activity in response to faces showed good test-retest reliability (ICC = 0.71). All other test-retest reliabilities were low (0.01 ≤ ICC ≤ 0.35). Reliabilities varied only slightly with increasing numbers of trials, with no consistent increase in ICCs. Test-retest reliabilities for left-handed participants (0.28 ≤ ICC ≤0.66) were generally somewhat higher than for right-handed participants (-0.13 ≤ ICC ≤0.75), but not statistically significant.ConclusionOur study shows good test-retest reliability for fusiform facer area activity in response to faces, but low test-retest reliability for other contrasts and areas.
| S-EPMC6422824 | biostudies-literature
Project description:BACKGROUND:Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed. OBJECTIVES:We aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability. METHODS:A search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC. RESULTS:From an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a "poor" ICC of 0.29 (95% CI?=?0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage. CONCLUSION:This study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.
| S-EPMC6907736 | biostudies-literature
Project description:While structural network analysis consolidated the hypothesis of cerebral small vessel disease (SVD) being a disconnection syndrome, little is known about functional changes on the level of brain networks. In patients with genetically defined SVD (CADASIL, n = 41) and sporadic SVD (n = 46), we independently tested the hypothesis that functional networks change with SVD burden and mediate the effect of disease burden on cognitive performance, in particular slowing of processing speed. We further determined test-retest reliability of functional network measures in sporadic SVD patients participating in a high-frequency (monthly) serial imaging study (RUN DMC-InTENse, median: 8 MRIs per participant). Functional networks for the whole brain and major subsystems (i.e., default mode network, DMN; fronto-parietal task control network, FPCN; visual network, VN; hand somatosensory-motor network, HSMN) were constructed based on resting-state multi-band functional MRI. In CADASIL, global efficiency (a graph metric capturing network integration) of the DMN was lower in patients with high disease burden (standardized beta = -.44; p [corrected] = .035) and mediated the negative effect of disease burden on processing speed (indirect path: std. beta = -.20, p = .047; direct path: std. beta = -.19, p = .25; total effect: std. beta = -.39, p = .02). The corresponding analyses in sporadic SVD showed no effect. Intraclass correlations in the high-frequency serial MRI dataset of the sporadic SVD patients revealed poor test-retest reliability and analysis of individual variability suggested an influence of age, but not disease burden, on global efficiency. In conclusion, our results suggest that changes in functional connectivity networks mediate the effect of SVD-related brain damage on cognitive deficits. However, limited reliability of functional network measures, possibly due to age-related comorbidities, impedes the analysis in elderly SVD patients.
| S-EPMC7294060 | biostudies-literature
Project description:BackgroundThe combination of resting-state functional MRI (R-fMRI) technique and graph theoretical approaches has emerged as a promising tool for characterizing the topological organization of brain networks, that is, functional connectomics. In particular, the construction and analysis of high-resolution brain connectomics at a voxel scale are important because they do not require prior regional parcellations and provide finer spatial information about brain connectivity. However, the test-retest reliability of voxel-based functional connectomics remains largely unclear.AimsThis study tended to investigate both short-term (∼20 min apart) and long-term (6 weeks apart) test-retest (TRT) reliability of graph metrics of voxel-based brain networks.MethodsBased on graph theoretical approaches, we analyzed R-fMRI data from 53 young healthy adults who completed two scanning sessions (session 1 included two scans 20 min apart; session 2 included one scan that was performed after an interval of ∼6 weeks).ResultsThe high-resolution networks exhibited prominent small-world and modular properties and included functional hubs mainly located at the default-mode, salience, and executive control systems. Further analysis revealed that test-retest reliabilities of network metrics were sensitive to the scanning orders and intervals, with fair to excellent long-term reliability between Scan 1 and Scan 3 and lower reliability involving Scan 2. In the long-term case (Scan 1 and Scan 3), most network metrics were generally test-retest reliable, with the highest reliability in global metrics in the clustering coefficient and in the nodal metrics in nodal degree and efficiency.ConclusionWe showed high test-retest reliability for graph properties in the high-resolution functional connectomics, which provides important guidance for choosing reliable network metrics and analysis strategies in future studies.
| S-EPMC6493187 | biostudies-literature