Dataset Information

Multisite reliability and repeatability of an advanced brain MRI protocol.

ABSTRACT: BACKGROUND:MRI is the imaging modality of choice for diagnosis and intervention assessment in neurological disease. Its full potential has not been realized due in part to challenges in harmonizing advanced techniques across multiple sites. PURPOSE:To develop a method for the assessment of reliability and repeatability of advanced multisite-multisession neuroimaging studies and specifically to assess the reliability of an advanced MRI protocol, including multiband fMRI and diffusion tensor MRI, in a multisite setting. STUDY TYPE:Prospective. POPULATION:Twice repeated measurement of a single subject with stable relapsing-remitting multiple sclerosis (MS) at seven institutions. FIELD STRENGTH/SEQUENCE:A 3 T MRI protocol included higher spatial resolution anatomical scans, a variable flip-angle longitudinal relaxation rate constant (R1 ? 1/T1 ) measurement, quantitative magnetization transfer imaging, diffusion tensor imaging, and a resting-state fMRI (rsFMRI) series. ASSESSMENT:Multiple methods of assessing intrasite repeatability and intersite reliability were evaluated for imaging metrics derived from each sequence. STATISTICAL TESTS:Student's t-test, Pearson's r, and intraclass correlation coefficient (ICC) (2,1) were employed to assess repeatability and reliability. Two new statistical metrics are introduced that frame reliability and repeatability in the respective units of the measurements themselves. RESULTS:Intrasite repeatability was excellent for quantitative R1 , magnetization transfer ratio (MTR), and diffusion-weighted imaging (DWI) based metrics (r > 0.95). rsFMRI metrics were less repeatable (r = 0.8). Intersite reliability was excellent for R1 , MTR, and DWI (ICC >0.9), and moderate for rsFMRI metrics (ICC?0.4). DATA CONCLUSION:From most reliable to least, using a new reliability metric introduced here, MTR > R1 > DWI > rsFMRI; for repeatability, MTR > DWI > R1 > rsFMRI. A graphical method for at-a-glance assessment of reliability and repeatability, effect sizes, and outlier identification in multisite-multisession neuroimaging studies is introduced. LEVEL OF EVIDENCE:1 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:878-888.

SUBMITTER: Schwartz DL

PROVIDER: S-EPMC6636359 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multisite reliability and repeatability of an advanced brain MRI protocol.

Schwartz Daniel L DL Tagge Ian I Powers Katherine K Ahn Sinyeob S Bakshi Rohit R Calabresi Peter A PA Todd Constable R R Grinstead John J Henry Roland G RG Nair Govind G Papinutto Nico N Pelletier Daniel D Shinohara Russell R Oh Jiwon J Reich Daniel S DS Sicotte Nancy L NL Rooney William D WD

Journal of magnetic resonance imaging : JMRI 20190116 3

<h4>Background</h4>MRI is the imaging modality of choice for diagnosis and intervention assessment in neurological disease. Its full potential has not been realized due in part to challenges in harmonizing advanced techniques across multiple sites.<h4>Purpose</h4>To develop a method for the assessment of reliability and repeatability of advanced multisite-multisession neuroimaging studies and specifically to assess the reliability of an advanced MRI protocol, including multiband fMRI and diffusi ...[more]

PMID: 30652391

Similar Datasets

Project description:Reliability of data has become a major concern in the course of the reproducibility crisis. Especially when studying animal behavior, confounding factors such as novelty of the test apparatus can lead to a wide variability of data which may mask treatment effects and consequently lead to misinterpretation. Habituation to the test situation is a common practice to circumvent novelty induced increases in variance and to improve the reliability of the respective measurements. However, there is a lack of published empirical knowledge regarding reasonable habituation procedures and a method validation seems to be overdue. This study aimed at setting up a simple strategy to increase reliability of behavioral data measured in a familiar test apparatus. Therefore, exemplary data from mice tested in an Open Field (OF) arena were used to elucidate the potential of habituation and how reliability of measures can be confirmed by means of a repeatability analysis using the software R. On seven consecutive days, male C57BL/6J, BALB/cJ and 129S1/SvImJ mice were tested in an OF arena once daily and individual mouse behavior was recorded. A repeatability analysis was conducted with regard to repeated trials of habituation. Our data analysis revealed that monitoring animal behavior during habituation is important to determine when individual differences of the measurements are stable. Repeatability values from distance travelled and average activity increased over the habituation period, revealing that around 60% of the variance of the data can be explained by individual differences between mice. The first day of habituation was significantly different from the following 6 days. A three-day habituation period appeared to be sufficient in this study. Overall, these results emphasize the importance of habituation and in depth analysis of habituation data to define the correct starting point of the experiment for improving the reliability and reproducibility of experimental data.

Project description:Recent years have witnessed an increasing number of multisite MRI functional connectivity (fcMRI) studies. While multisite studies provide an efficient way to accelerate data collection and increase sample sizes, especially for rare clinical populations, any effects of site or MRI scanner could ultimately limit power and weaken results. Little data exists on the stability of functional connectivity measurements across sites and sessions. In this study, we assess the influence of site and session on resting state functional connectivity measurements in a healthy cohort of traveling subjects (8 subjects scanned twice at each of 8 sites) scanned as part of the North American Prodrome Longitudinal Study (NAPLS). Reliability was investigated in three types of connectivity analyses: (1) seed-based connectivity with posterior cingulate cortex (PCC), right motor cortex (RMC), and left thalamus (LT) as seeds; (2) the intrinsic connectivity distribution (ICD), a voxel-wise connectivity measure; and (3) matrix connectivity, a whole-brain, atlas-based approach to assessing connectivity between nodes. Contributions to variability in connectivity due to subject, site, and day-of-scan were quantified and used to assess between-session (test-retest) reliability in accordance with Generalizability Theory. Overall, no major site, scanner manufacturer, or day-of-scan effects were found for the univariate connectivity analyses; instead, subject effects dominated relative to the other measured factors. However, summaries of voxel-wise connectivity were found to be sensitive to site and scanner manufacturer effects. For all connectivity measures, although subject variance was three times the site variance, the residual represented 60-80% of the variance, indicating that connectivity differed greatly from scan to scan independent of any of the measured factors (i.e., subject, site, and day-of-scan). Thus, for a single 5min scan, reliability across connectivity measures was poor (ICC=0.07-0.17), but increased with increasing scan duration (ICC=0.21-0.36 at 25min). The limited effects of site and scanner manufacturer support the use of multisite studies, such as NAPLS, as a viable means of collecting data on rare populations and increasing power in univariate functional connectivity studies. However, the results indicate that aggregation of fcMRI data across longer scan durations is necessary to increase the reliability of connectivity estimates at the single-subject level.

Project description:Purpose:Researchers have long noted speech recognition variability that is not explained by the pure-tone audiogram. Previous work (Souza, Wright, Blackburn, Tatman, & Gallun, 2015) demonstrated that a small number of listeners with sensorineural hearing loss utilized different types of acoustic cues to identify speechlike stimuli, specifically the extent to which the participant relied upon spectral (or temporal) information for identification. Consistent with recent calls for data rigor and reproducibility, the primary aims of this study were to replicate the pattern of cue use in a larger cohort and to verify stability of the cue profiles over time. Method:Cue-use profiles were measured for adults with sensorineural hearing loss using a syllable identification task consisting of synthetic speechlike stimuli in which spectral and temporal dimensions were manipulated along continua. For the first set, a static spectral shape varied from alveolar to palatal, and a temporal envelope rise time varied from affricate to fricative. For the second set, formant transitions varied from labial to alveolar and a temporal envelope rise time varied from approximant to stop. A discriminant feature analysis was used to determine to what degree spectral and temporal information contributed to stimulus identification. A subset of participants completed a 2nd visit using the same stimuli and procedures. Results:When spectral information was static, most participants were more influenced by spectral than by temporal information. When spectral information was dynamic, participants demonstrated a balanced distribution of cue-use patterns, with nearly equal numbers of individuals influenced by spectral or temporal cues. Individual cue profile was repeatable over a period of several months. Conclusion:In combination with previously published data, these results indicate that listeners with sensorineural hearing loss are influenced by different cues to identify speechlike sounds and that those patterns are stable over time.

Dataset Information

Multisite reliability and repeatability of an advanced brain MRI protocol.

Publications

Multisite reliability and repeatability of an advanced brain MRI protocol.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets