Dataset Information

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

ABSTRACT: BACKGROUND:Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed. OBJECTIVES:We aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability. METHODS:A search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC. RESULTS:From an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a "poor" ICC of 0.29 (95% CI?=?0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage. CONCLUSION:This study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.

SUBMITTER: Noble S

PROVIDER: S-EPMC6907736 | biostudies-literature | 2019 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

Noble Stephanie S Scheinost Dustin D Constable R Todd RT

NeuroImage 20190905

<h4>Background</h4>Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed.<h4>Objectives</h4>We aimed to summarize the exi ...[more]

PMID: 31494250

Similar Datasets

Project description:Resting functional MRI studies of the infant brain are increasingly becoming an important tool in developmental neuroscience. Whereas the test-retest reliability of functional connectivity (FC) measures derived from resting fMRI data have been characterized in the adult and child brain, similar assessments have not been conducted in infants. In this study, we examined the intra-session test-retest reliability of FC measures from 119 infant brain MRI scans from four neurodevelopmental studies. We investigated edge-level and subject-level reliability within one MRI session (between and within runs) measured by the Intraclass correlation coefficient (ICC). First, using an atlas-based approach, we examined whole-brain connectivity as well as connectivity within two common resting fMRI networks - the default mode network (DMN) and the sensorimotor network (SMN). Second, we examined the influence of run duration, study site, and scanning manufacturer (e.g., Philips and General Electric) on ICCs. Lastly, we tested spatial similarity using the Jaccard Index from networks derived from independent component analysis (ICA). Consistent with resting fMRI studies from adults, our findings indicated poor edge-level reliability (ICC = 0.14-0.18), but moderate-to-good subject-level intra-session reliability for whole-brain, DMN, and SMN connectivity (ICC = 0.40-0.78). We also found significant effects of run duration, site, and scanning manufacturer on reliability estimates. Some ICA-derived networks showed strong spatial reproducibility (e.g., DMN, SMN, and Visual Network), and were labelled based on their spatial similarity to analogous networks measured in adults. These networks were reproducibly found across different study sites. However, other ICA-networks (e.g. Executive Control Network) did not show strong spatial reproducibility, suggesting that the reliability and/or maturational course of functional connectivity may vary by network. In sum, our findings suggest that developmental scientists may be on safe ground examining the functional organization of some major neural networks (e.g. DMN and SMN), but judicious interpretation of functional connectivity is essential to its ongoing success.

Project description:The reliability of magnetoencephalography (MEG) resting-state functional connectivity in schizophrenia (SZ) is unknown as previous research has focused on healthy controls (HC). Here, we examined reliability in 26 participants (13-SZ, 13-HC). Eyes opened and eyes closed resting-state data were collected on 4 separate occasions during 2 visits, 1 week apart. For source modeling, we used minimum norm software to apply dynamic statistical parametric mapping. Source analyses compared the following functional connectivity metrics from each data run: coherence (coh), imaginary coherence (imcoh), pairwise phase consistency (ppc), phase-locking value (plv), phase lag index (pli), weighted phase lag index (wpli), and weighted phase lag index debiased (wpli2). Intraclass correlation coefficients (ICCs) were calculated for whole brain, network, and network pair averages. For reliability, ICCs above 0.75 = excellent, above 0.60 = good, above 0.40 = fair, and below 0.40 = poor reliability. We found the reliability of these metrics varied greatly depending on frequency band, network, network pair, and participant group examined. Broadband (1-58 Hz) whole brain averages in both HC and SZ showed excellent reliability for wpli2, and good to fair reliability for ppc, plv, and coh. Broadband network averages showed excellent to good reliability across 1 hour and 1 week for coh, imcoh, ppc, plv, wpli within default mode, cognitive control, and visual networks in HC, while the same metrics had excellent to fair reliability in SZ. Regional network pair averages showed good to fair reliability for coh, ppc, plv within default mode, cognitive control and visual network pairs in HC and SZ. In general, HC had higher reliability compared to SZ, and the default mode, cognitive control, and visual networks had higher reliability compared to somatosensory and auditory networks. Similar reliability levels occurred for both eyes opened and eyes closed resting-states for most metrics. The functional connectivity metrics of coh, ppc, and plv performed best across 1 hour and 1 week in HC and SZ. We also found that SZ had reduced coh, plv, and ppc in the dmn average and pair values indicating dysconnectivity in SZ. These findings encourage collecting both eyes opened and eyes closed resting-state MEG, while demonstrating that clinical populations may differ in reliability.

Project description:Functional connectivity (FC) has become a leading method for resting-state functional magnetic resonance imaging (rs-fMRI) analysis. However, the majority of the previous studies utilized pairwise, temporal synchronization-based FC. Recently, high-order FC (HOFC) methods were proposed with the idea of computing "correlation of correlations" to capture high-level, more complex associations among the brain regions. There are two types of HOFC. The first type is topographical profile similarity-based HOFC (tHOFC) and its variant, associated HOFC (aHOFC), for capturing different levels of HOFC. Instead of measuring the similarity of the original rs-fMRI signals with the traditional FC (low-order FC, or LOFC), tHOFC measures the similarity of LOFC profiles (i.e., a set of LOFC values between a region and all other regions) between each pair of brain regions. The second type is dynamics-based HOFC (dHOFC) which defines the quadruple relationship among every four brain regions by first calculating two pairwise dynamic LOFC "time series" and then measuring their temporal synchronization (i.e., temporal correlation of the LOFC fluctuations, not the BOLD fluctuations). Applications have shown the superiority of HOFC in both disease biomarker detection and individualized diagnosis than LOFC. However, no study has been carried out for the assessment of test-retest reliability of different HOFC metrics. In this paper, we systematically evaluate the reliability of the two types of HOFC methods using test-retest rs-fMRI data from 25 (12 females, age 24.48 ± 2.55 years) young healthy adults with seven repeated scans (with interval = 3-8 days). We found that all HOFC metrics have satisfactory reliability, specifically (1) fair-to-good for tHOFC and aHOFC, and (2) fair-to-moderate for dHOFC with relatively strong connectivity strength. We further give an in-depth analysis of the biological meanings of each HOFC metric and highlight their differences compared to the LOFC from the aspects of cross-level information exchanges, within-/between-network connectivity, and modulatory connectivity. In addition, how the dynamic analysis parameter (i.e., sliding window length) affects dHOFC reliability is also investigated. Our study reveals unique functional associations characterized by the HOFC metrics. Guidance and recommendations for future applications and clinical research using HOFC are provided. This study has made a further step toward unveiling more complex human brain connectome.

Project description:Resting-state functional MRI (rs-fMRI) has emerged as a powerful tool for investigating brain functional connectivity (FC). Research in recent years has focused on assessing the reliability of FC across younger subjects within and between scan-sessions. Test-retest reliability in resting-state functional connectivity (RSFC) has not yet been examined in older adults. In this study, we investigated age-related differences in reliability and stability of RSFC across scans. In addition, we examined how global signal regression (GSR) affects RSFC reliability and stability. Three separate resting-state scans from 29 younger adults (18-35 yrs) and 26 older adults (55-85 yrs) were obtained from the International Consortium for Brain Mapping (ICBM) dataset made publically available as part of the 1000 Functional Connectomes project www.nitrc.org/projects/fcon_1000. 92 regions of interest (ROIs) with 5 cubic mm radius, derived from the default, cingulo-opercular, fronto-parietal and sensorimotor networks, were previously defined based on a recent study. Mean time series were extracted from each of the 92 ROIs from each scan and three matrices of z-transformed correlation coefficients were created for each subject, which were then used for evaluation of multi-scan reliability and stability. The young group showed higher reliability of RSFC than the old group with GSR (p-value = 0.028) and without GSR (p-value <0.001). Both groups showed a high degree of multi-scan stability of RSFC and no significant differences were found between groups. By comparing the test-retest reliability of RSFC with and without GSR across scans, we found significantly higher proportion of reliable connections in both groups without GSR, but decreased stability. Our results suggest that aging is associated with reduced reliability of RSFC which itself is highly stable within-subject across scans for both groups, and that GSR reduces the overall reliability but increases the stability in both age groups and could potentially alter group differences of RSFC.

Dataset Information

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

Publications

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets