Test-retest reproducibility of a multi-atlas automated segmentation tool on multimodality brain MRI.
Ontology highlight
ABSTRACT: Introduction
The increasing use of large sample sizes for population and personalized medicine requires high-throughput tools for imaging processing that can handle large amounts of data with diverse image modalities, perform a biologically meaningful information reduction, and result in comprehensive quantification. Exploring the reproducibility of these tools reveals the specific strengths and weaknesses that heavily influence the interpretation of results, contributing to transparence in science.Methods
We tested-retested the reproducibility of MRICloud, a free automated method for whole-brain, multimodal MRI segmentation and quantification, on two public, independent datasets of healthy adults.Results
The reproducibility was extremely high for T1-volumetric analysis, high for diffusion tensor images (DTI) (however, regionally variable), and low for resting-state fMRI.Conclusion
In general, the reproducibility of the different modalities was slightly superior to that of widely used software. This analysis serves as a normative reference for planning samples and for the interpretation of structure-based MRI studies.
SUBMITTER: Rezende TJR
PROVIDER: S-EPMC6790328 | biostudies-literature | 2019 Oct
REPOSITORIES: biostudies-literature
Publications
Test-retest reproducibility of a multi-atlas automated segmentation tool on multimodality brain MRI.
Rezende Thiago J R TJR Campos Brunno M BM Hsu Johnny J Li Yue Y Ceritoglu Can C Kutten Kwame K França Junior Marcondes C MC Mori Susumu S Miller Michael I MI Faria Andreia V AV
Brain and behavior 20190904 10
<h4>Introduction</h4>The increasing use of large sample sizes for population and personalized medicine requires high-throughput tools for imaging processing that can handle large amounts of data with diverse image modalities, perform a biologically meaningful information reduction, and result in comprehensive quantification. Exploring the reproducibility of these tools reveals the specific strengths and weaknesses that heavily influence the interpretation of results, contributing to transparence ...[more]
Similar Datasets
Project description:Accurate and reproducible automated segmentation of human hippocampal subfields is of interest to study their roles in cognitive functions and disease processes. Multispectral structural MRI methods have been proposed to improve automated hippocampal subfield segmentation accuracy, but the reproducibility in a multicentric setting is, to date, not well characterized. Here, we assessed test-retest reproducibility of FreeSurfer 6.0 hippocampal subfield segmentations using multispectral MRI analysis pipelines (22 healthy subjects scanned twice, a week apart, at four 3T MRI sites). The harmonized MRI protocol included two 3D-T1, a 3D-FLAIR, and a high-resolution 2D-T2. After within-session T1 averaging, subfield volumes were segmented using three pipelines with different multispectral data: two longitudinal ("long_T1s" and "long_T1s_FLAIR") and one cross-sectional ("long_T1s_FLAIR_crossT2"). Volume reproducibility was quantified in magnitude (reproducibility error-RE) and space (DICE coefficient). RE was lower in all hippocampal subfields, except for hippocampal fissure, using the longitudinal pipelines compared to long_T1s_FLAIR_crossT2 (average RE reduction of 0.4-3.6%). Similarly, the longitudinal pipelines showed a higher spatial reproducibility (1.1-7.8% of DICE improvement) in all hippocampal structures compared to long_T1s_FLAIR_crossT2. Moreover, long_T1s_FLAIR provided a small but significant RE improvement in comparison to long_T1s (p = 0.015), whereas no significant DICE differences were found. In addition, structures with volumes larger than 200 mm3 had better RE (1-2%) and DICE (0.7-0.95) than smaller structures. In summary, our study suggests that the most reproducible hippocampal subfield FreeSurfer segmentations are derived from a longitudinal pipeline using 3D-T1s and 3D-FLAIR. Adapting a longitudinal pipeline to include high-resolution 2D-T2 may lead to further improvements.
| S-EPMC7817563 | biostudies-literature
Project description:INTRODUCTION:Preclinical in vivo imaging requires precise and reproducible delineation of brain structures. Manual segmentation is time consuming and operator dependent. Automated segmentation as usually performed via single atlas registration fails to account for anatomo-physiological variability. We present, evaluate, and make available a multi-atlas approach for automatically segmenting rat brain MRI and extracting PET activies. METHODS:High-resolution 7T 2DT2 MR images of 12 Sprague-Dawley rat brains were manually segmented into 27-VOI label volumes using detailed protocols. Automated methods were developed with 7/12 atlas datasets, i.e. the MRIs and their associated label volumes. MRIs were registered to a common space, where an MRI template and a maximum probability atlas were created. Three automated methods were tested: 1/registering individual MRIs to the template, and using a single atlas (SA), 2/using the maximum probability atlas (MP), and 3/registering the MRIs from the multi-atlas dataset to an individual MRI, propagating the label volumes and fusing them in individual MRI space (propagation & fusion, PF). Evaluation was performed on the five remaining rats which additionally underwent [18F]FDG PET. Automated and manual segmentations were compared for morphometric performance (assessed by comparing volume bias and Dice overlap index) and functional performance (evaluated by comparing extracted PET measures). RESULTS:Only the SA method showed volume bias. Dice indices were significantly different between methods (PF>MP>SA). PET regional measures were more accurate with multi-atlas methods than with SA method. CONCLUSIONS:Multi-atlas methods outperform SA for automated anatomical brain segmentation and PET measure's extraction. They perform comparably to manual segmentation for FDG-PET quantification. Multi-atlas methods are suitable for rapid reproducible VOI analyses.
| S-EPMC4201469 | biostudies-literature
Project description:There are two popular approaches for automated white matter parcellation using diffusion MRI tractography, including fiber clustering strategies that group white matter fibers according to their geometric trajectories and cortical-parcellation-based strategies that focus on the structural connectivity among different brain regions of interest. While multiple studies have assessed test-retest reproducibility of automated white matter parcellations using cortical-parcellation-based strategies, there are no existing studies of test-retest reproducibility of fiber clustering parcellation. In this work, we perform what we believe is the first study of fiber clustering white matter parcellation test-retest reproducibility. The assessment is performed on three test-retest diffusion MRI datasets including a total of 255 subjects across genders, a broad age range (5-82 years), health conditions (autism, Parkinson's disease and healthy subjects), and imaging acquisition protocols (three different sites). A comprehensive evaluation is conducted for a fiber clustering method that leverages an anatomically curated fiber clustering white matter atlas, with comparison to a popular cortical-parcellation-based method. The two methods are compared for the two main white matter parcellation applications of dividing the entire white matter into parcels (i.e., whole brain white matter parcellation) and identifying particular anatomical fiber tracts (i.e., anatomical fiber tract parcellation). Test-retest reproducibility is measured using both geometric and diffusion features, including volumetric overlap (wDice) and relative difference of fractional anisotropy. Our experimental results in general indicate that the fiber clustering method produced more reproducible white matter parcellations than the cortical-parcellation-based method.
| S-EPMC6548665 | biostudies-literature
Project description:Quantification of tissue magnetic susceptibility using MRI offers a non-invasive measure of important tissue components in the brain, such as iron and myelin, potentially providing valuable information about normal and pathological conditions during aging. Despite many advances made in recent years on imaging techniques of quantitative susceptibility mapping (QSM), accurate and robust automated segmentation tools for QSM images that can help generate universal and sharable susceptibility measures in a biologically meaningful set of structures are still not widely available. In the present study, we developed an automated process to segment brain nuclei and quantify tissue susceptibility in these regions based on a susceptibility multi-atlas library, consisting of 10 atlases with T1-weighted images, gradient echo (GRE) magnitude images and QSM images of brains with different anatomic patterns. For each atlas in this library, 10 regions of interest in iron-rich deep gray matter structures that are better defined by QSM contrast were manually labeled, including caudate, putamen, globus pallidus internal/external, thalamus, pulvinar, subthalamic nucleus, substantia nigra, red nucleus and dentate nucleus in both left and right hemispheres. We then tested different pipelines using different combinations of contrast channels to bring the set of labels from the multi-atlases to each target brain and compared them with the gold standard manual delineation. The results showed that the segmentation accuracy using dual contrasts QSM/T1 pipeline outperformed other dual-contrast or single-contrast pipelines. The dice values of 0.77 ± 0.09 using the QSM/T1 multi-atlas pipeline rivaled with the segmentation reliability obtained from multiple evaluators with dice values of 0.79 ± 0.07 and gave comparable or superior performance in segmenting subcortical nuclei in comparison with standard FSL FIRST or recent multi-atlas package of volBrain. The segmentation performance of the QSM/T1 multi-atlas was further tested on QSM images acquired using different acquisition protocols and platforms and showed good reliability and reproducibility with average dice of 0.79 ± 0.08 to manual labels and 0.89 ± 0.04 in an inter-protocol manner. The extracted quantitative magnetic susceptibility values in the deep gray matter nuclei also correlated well between different protocols with inter-protocol correlation constants all larger than 0.97. Such reliability and performance was ultimately validated in an external dataset acquired at another study site with consistent susceptibility measures obtained using the QSM/T1 multi-atlas approach in comparison to those using manual delineation. In summary, we designed a susceptibility multi-atlas tool for automated and reliable segmentation of QSM images and for quantification of magnetic susceptibilities. It is publicly available through our cloud-based platform (www.mricloud.org). Further improvement on the performance of this multi-atlas tool is expected by increasing the number of atlases in the future.
| S-EPMC6464637 | biostudies-literature
Project description:Multi-atlas segmentation has been shown to be highly robust and accurate across an extraordinary range of potential applications. However, it is limited to the segmentation of structures that are anatomically consistent across a large population of potential target subjects (i.e., multi-atlas segmentation is limited to "in-atlas" applications). Herein, the authors propose a technique to determine the likelihood that a multi-atlas segmentation estimate is representative of the problem at hand, and, therefore, identify anomalous regions that are not well represented within the atlases.The authors derive a technique to estimate the out-of-atlas (OOA) likelihood for every voxel in the target image. These estimated likelihoods can be used to determine and localize the probability of an abnormality being present on the target image.Using a collection of manually labeled whole-brain datasets, the authors demonstrate the efficacy of the proposed framework on two distinct applications. First, the authors demonstrate the ability to accurately and robustly detect malignant gliomas in the human brain-an aggressive class of central nervous system neoplasms. Second, the authors demonstrate how this OOA likelihood estimation process can be used within a quality control context for diffusion tensor imaging datasets to detect large-scale imaging artifacts (e.g., aliasing and image shading).The proposed OOA likelihood estimation framework shows great promise for robust and rapid identification of brain abnormalities and imaging artifacts using only weak dependencies on anomaly morphometry and appearance. The authors envision that this approach would allow for application-specific algorithms to focus directly on regions of high OOA likelihood, which would (1) reduce the need for human intervention, and (2) reduce the propensity for false positives. Using the dual perspective, this technique would allow for algorithms to focus on regions of normal anatomy to ascertain image quality and adapt to image appearance characteristics.
| S-EPMC3625241 | biostudies-other
Project description:Quantitative size, shape, and texture features derived from computed tomographic (CT) images may be useful as predictive, prognostic, or response biomarkers in non-small cell lung cancer (NSCLC). However, to be useful, such features must be reproducible, non-redundant, and have a large dynamic range. We developed a set of quantitative three-dimensional (3D) features to describe segmented tumors and evaluated their reproducibility to select features with high potential to have prognostic utility. Thirty-two patients with NSCLC were subjected to unenhanced thoracic CT scans acquired within 15 min of each other under an approved protocol. Primary lung cancer lesions were segmented using semi-automatic 3D region growing algorithms. Following segmentation, 219 quantitative 3D features were extracted from each lesion, corresponding to size, shape, and texture, including features in transformed spaces (laws, wavelets). The most informative features were selected using the concordance correlation coefficient across test-retest, the biological range and a feature independence measure. There were 66 (30.14%) features with concordance correlation coefficient???0.90 across test-retest and acceptable dynamic range. Of these, 42 features were non-redundant after grouping features with R (2) Bet???0.95. These reproducible features were found to be predictive of radiological prognosis. The area under the curve (AUC) was 91% for a size-based feature and 92% for the texture features (runlength, laws). We tested the ability of image features to predict a radiological prognostic score on an independent NSCLC (39 adenocarcinoma) samples, the AUC for texture features (runlength emphasis, energy) was 0.84 while the conventional size-based features (volume, longest diameter) was 0.80. Test-retest and correlation analyses have identified non-redundant CT image features with both high intra-patient reproducibility and inter-patient biological range. Thus making the case that quantitative image features are informative and prognostic biomarkers for NSCLC.
| S-EPMC4391075 | biostudies-literature
Project description:Test-retest of automated image segmentation algorithms (FSL FAST, FSL FIRST, and FREESURFER) are computed on magnetic resonance images from 12 unsedated children aged 9.4±2.6 years ([min,max] = [6.5 years, 13.8 years]) using different approaches to motion correction (prospective versus retrospective). The prospective technique, PROMO MPRAGE, dynamically estimates motion using specially acquired navigator images and adjusts the remaining acquisition accordingly, whereas the retrospective technique, MPnRAGE, uses a self-navigation property to retrospectively estimate and account for motion during image reconstruction. To increase the likelihood and range of motions, participants heads were not stabilized with padding during repeated scans. When motion was negligible both techniques had similar performance. When motion was not negligible, the automated image segmentation and anatomical labeling software tools showed the most consistent performance with the retrospectively corrected MPnRAGE technique (≥80% volume overlaps for 15 of 16 regions for FIRST and FREESURFER, with greater than 90% volume overlaps for 12 regions with FIRST and 11 regions with FREESURFER). Prospectively corrected MPRAGE with linear view-ordering also demonstrated lower performance than MPnRAGE without retrospective motion correction.
| S-EPMC7056555 | biostudies-literature
Project description:Using semi-automated software simplifies quantitative analysis of the visible burden of disease on whole-body MRI diffusion-weighted images. To establish the intra- and inter-observer reproducibility of apparent diffusion coefficient (ADC) measures, we retrospectively analyzed data from 20 patients with bone metastases from breast (BCa; n = 10; aged 62.3 ± 14.8) or prostate cancer (PCa; n = 10; aged 67.4 ± 9.0) who had undergone examinations at two timepoints, before and after hormone-therapy. Four independent observers processed all images twice, first segmenting the entire skeleton on diffusion-weighted images, and then isolating bone metastases via ADC histogram thresholding (ADC: 650-1400 µm2/s). Dice Similarity, Bland-Altman method, and Intraclass Correlation Coefficient were used to assess reproducibility. Inter-observer Dice similarity was moderate (0.71) for women with BCa and poor (0.40) for men with PCa. Nonetheless, the limits of agreement of the mean ADC were just ±6% for women with BCa and ±10% for men with PCa (mean ADCs: 941 and 999 µm2/s, respectively). Inter-observer Intraclass Correlation Coefficients of the ADC histogram parameters were consistently greater in women with BCa than in men with PCa. While scope remains for improving consistency of the volume segmented, the observer-dependent variability measured in this study was appropriate to distinguish the clinically meaningful changes of ADC observed in patients responding to therapy, as changes of at least 25% are of interest.
| S-EPMC7998160 | biostudies-literature
Project description:Background and purposeMicrostructure imaging with advanced diffusion MRI (dMRI) techniques have shown increased sensitivity and specificity to microstructural changes in various disease and injury models. Oscillating gradient spin echo (OGSE) dMRI, implemented by varying the oscillating gradient frequency, and microscopic anisotropy (μA) dMRI, implemented via tensor valued diffusion encoding, may provide additional insight by increasing sensitivity to smaller spatial scales and disentangling fiber orientation dispersion from true microstructural changes, respectively. The aims of this study were to characterize the test-retest reproducibility of in vivo OGSE and μA dMRI metrics in the mouse brain at 9.4 Tesla and provide estimates of required sample sizes for future investigations.MethodsTwelve adult C57Bl/6 mice were scanned twice (5 days apart). Each imaging session consisted of multifrequency OGSE and μA dMRI protocols. Metrics investigated included μA, linear diffusion kurtosis, isotropic diffusion kurtosis, and the diffusion dispersion rate (Λ), which explores the power-law frequency dependence of mean diffusivity. The dMRI metric maps were analyzed with mean region-of-interest (ROI) and whole brain voxel-wise analysis. Bland-Altman plots and coefficients of variation (CV) were used to assess the reproducibility of OGSE and μA metrics. Furthermore, we estimated sample sizes required to detect a variety of effect sizes.ResultsBland-Altman plots showed negligible biases between test and retest sessions. ROI-based CVs revealed high reproducibility for most metrics (CVs < 15%). Voxel-wise CV maps revealed high reproducibility for μA (CVs ~ 10%), but low reproducibility for OGSE metrics (CVs ~ 50%).ConclusionMost of the μA dMRI metrics are reproducible in both ROI-based and voxel-wise analysis, while the OGSE dMRI metrics are only reproducible in ROI-based analysis. Given feasible sample sizes (10-15), μA metrics and OGSE metrics may provide sensitivity to subtle microstructural changes (4-8%) and moderate changes (> 6%), respectively.
| S-EPMC8570471 | biostudies-literature
Project description:Three-dimensional printing has an increasing number of clinical applications in pediatric cardiology. Time required for dataset segmentation and conversion to stereolithography (STL) format remains a significant limitation. We investigated the impact of semi-automated cardiovascular-specific segmentation software on time and reproducibility of segmentation. Magnetic resonance angiograms (MRAs) of 19 patients undergoing intervention for right ventricular outflow lesions were segmented to demonstrate the right heart. STLs were created by two independent clinicians using semi-automated cardiovascular segmentation (SAS) and traditional manual segmentation (MS). Time was recorded and geometric STL disagreement was determined (0 % = no disagreement, 100 % = complete disagreement). MRA datasets were categorized as clean when only right heart structures were present in the MRA, or contaminated when left heart structures were also present and required removal. Eighteen (seven clean and 11 contaminated) cases were successfully segmented with both methods. Time to STL for clean datasets was faster with MS than SAS [median 209 s (IQR 192-252) vs. 296 s (272-317), p = 0.018] while contaminated datasets were faster with SAS [455 s (384-561) vs. 866 s (310-1429), p = 0.033]. Interobserver STL geometric disagreement was significantly lower using SAS than MS overall (0.70 ± 1.15 % vs. 1.31 ± 1.52 %, p = 0.030), and for the contaminated subset (0.81 ± 1.08 % vs. 1.75 ± 1.57 %, p = 0.036). Most geometric disagreement occurred at areas where left heart contamination was removed. Semi-automated segmentation was faster and more reproducible for contaminated datasets, while MS was faster but equally reproducible for clean datasets. Semi-automated segmentation methods are preferable for contaminated datasets and continued refinement of these tools should be supported.
| S-EPMC5562952 | biostudies-other