Dataset Information

Reproducibility and Bias in Healthy Brain Segmentation: Comparison of Two Popular Neuroimaging Platforms.

ABSTRACT: We evaluated and compared the performance of two popular neuroimaging processing platforms: Statistical Parametric Mapping (SPM) and FMRIB Software Library (FSL). We focused on comparing brain segmentations using Kirby21, a magnetic resonance imaging (MRI) replication study with 21 subjects and two scans per subject conducted only a few hours apart. We tested within- and between-platform segmentation reliability both at the whole brain and in 10 regions of interest (ROIs). For a range of fixed probability thresholds we found no differences between-scans within-platform, but large differences between-platforms. We have also found very large differences between- and within-platforms when probability thresholds were changed. A randomized blinded reader study indicated that: (1) SPM and FSL performed well in terms of gray matter segmentation; (2) SPM and FSL performed poorly in terms of white matter segmentation; and (3) FSL slightly outperformed SPM in terms of CSF segmentation. We also found that tissue class probability thresholds can have profound effects on segmentation results. We conclude that the reproducibility of neuroimaging studies depends on the neuroimaging software-processing platform and tissue probability thresholds. Our results suggest that probability thresholds may not be comparable across platforms and consistency of results may be improved by estimating a probability threshold correspondence function between SPM and FSL.

SUBMITTER: Tudorascu DL

PROVIDER: S-EPMC5101202 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reproducibility and Bias in Healthy Brain Segmentation: Comparison of Two Popular Neuroimaging Platforms.

Tudorascu Dana L DL Karim Helmet T HT Maronge Jacob M JM Alhilali Lea L Fakhran Saeed S Aizenstein Howard J HJ Muschelli John J Crainiceanu Ciprian M CM

Frontiers in neuroscience 20161109

We evaluated and compared the performance of two popular neuroimaging processing platforms: Statistical Parametric Mapping (SPM) and FMRIB Software Library (FSL). We focused on comparing brain segmentations using Kirby21, a magnetic resonance imaging (MRI) replication study with 21 subjects and two scans per subject conducted only a few hours apart. We tested within- and between-platform segmentation reliability both at the whole brain and in 10 regions of interest (ROIs). For a range of fixed p ...[more]

PMID: 27881948

Similar Datasets

Project description:BackgroundHemodynamic aorta parameters can be derived from 4D flow MRI, but this requires lumen segmentation. In both commercially available and research 4D flow MRI software tools, lumen segmentation is mostly (semi-)automatically performed and subsequently manually improved by an observer. Since the segmentation variability, together with 4D flow MRI data and image processing algorithms, will contribute to the reproducibility of patient-specific flow properties, the observer's lumen segmentation reproducibility and repeatability needs to be assessed.PurposeTo determine the interexamination, interobserver reproducibility, and intraobserver repeatability of aortic lumen segmentation on 4D flow MRI.Study typeProspective and retrospective.PopulationA healthy volunteer cohort of 10 subjects who underwent 4D flow MRI twice. Also, a clinical cohort of six subjects who underwent 4D flow MRI once.Field strength/sequence3T; time-resolved three-directional and 3D velocity-encoded sequence (4D flow MRI).AssessmentThe thoracic aorta was segmented on the 4D flow MRI in five systolic phases. By positioning six planes perpendicular to a segmentation's centerline, the aorta was divided into five segments. The volume, surface area, centerline length, maximal diameter, and curvature radius were determined for each segment.Statistical testsTo assess the reproducibility, the coefficient of variation (COV), Pearson correlation coefficient (r), and intraclass correlation coefficient (ICC) were calculated.ResultsThe interexamination and interobserver reproducibility and intraobserver repeatability were comparable for each parameter. For both cohorts there was very good reproducibility and repeatability for volume, surface area, and centerline length (COV = 10-32%, r = 0.54-0.95 and ICC = 0.65-0.99), excellent reproducibility and repeatability for maximal diameter (COV = 3-11%, r = 0.94-0.99, ICC = 0.94-0.99), and good reproducibility and repeatability for curvature radius (COV = 25-62%, r = 0.73-0.95, ICC = 0.84-0.97).Data conclusionThis study demonstrated no major reproducibility and repeatability limitations for 4D flow MRI aortic lumen segmentation.Level of evidence3 TECHNICAL EFFICACY STAGE: 2.

Project description:BackgroundThe inter-scanner reproducibility of brain volumetry is important in multi-site neuroimaging studies, where the reliability of automated brain segmentation (ABS) tools plays an important role. This study aimed to evaluate the influence of ABS tools on the consistency and reproducibility of the quantified brain volumetry from different scanners.MethodsWe included fifteen healthy volunteers who were scanned with 3D isotropic brain T1-weighted sequence on three different 3.0 Tesla MRI scanners (GE, Siemens and Philips). For each individual, the time span between image acquisitions on different scanners was limited to 1 h. All the T1-weighted images were processed with FreeSurfer v6.0, FSL v5.0 and AccuBrain® with default settings to obtain volumetry of brain tissues (e.g. gray matter) and substructures (e.g. basal ganglia structures) if available. Coefficient of variation (CV) was calculated to test inter-scanner variability in brain volumetry of various structures as quantified by these ABS tools.ResultsThe mean inter-scanner CV values per brain structure among three MRI scanners ranged from 6.946 to 12.29% (mean, 9.577%) for FreeSurfer, 7.245 to 20.98% (mean, 12.60%) for FSL and 1.348 to 8.800% (mean value, 3.546%) for AccuBrain®. In addition, AccuBrain® and FreeSurfer achieved the lowest mean values of region-specific CV between GE and Siemens scanners (from 0.818 to 5.958% for AccuBrain®, and from 0.903 to 7.977% for FreeSurfer), while FSL-FIRST had the lowest mean values of region-specific CV between GE and Philips scanners (from 2.603 to 16.310%). AccuBrain® also had the lowest mean values of region-specific CV between Siemens and Philips scanners (from 1.138 to 6.615%).ConclusionThere is a large discrepancy in the inter-scanner reproducibility of brain volumetry when using different processing software. Image acquisition protocols and selection of ABS tool for brain volumetry quantification have impact on the robustness of results in multi-site studies.

Project description:PurposeInvestigate reproducibility of two segmentation methods for multicompartment dosimetry, including normal tissue absorbed dose (NTAD) and tumour absorbed dose (TAD), in hepatocellular carcinoma patients treated with yttrium-90 (90Y) glass microspheres.MethodsTARGET was a retrospective investigation in 209 patients with < 10 tumours per lobe and at least one tumour ≥ 3 cm ± portal vein thrombosis. Dosimetry was compared using two distinct segmentation methods: anatomic (CT/MRI-based) and count threshold-based on pre-procedural 99mTc-MAA SPECT. In a round robin substudy in 20 patients with ≤ 5 unilobar tumours, the inter-observer reproducibility of eight reviewers was evaluated by computing reproducibility coefficient (RDC) of volume and absorbed dose for whole liver, whole liver normal tissue, perfused normal tissue, perfused liver, total perfused tumour, and target lesion. Intra-observer reproducibility was based on second assessments in 10 patients ≥ 2 weeks later.Results99mTc-MAA segmentation calculated higher absorbed doses compared to anatomic segmentation (n = 209), 43.9% higher for TAD (95% limits of agreement [LoA]: - 49.0%, 306.2%) and 21.3% for NTAD (95% LoA: - 67.6%, 354.0%). For the round robin substudy (n = 20), inter-observer reproducibility was better for anatomic (RDC range: 1.17 to 3.53) than 99mTc-MAA SPECT segmentation (1.29 to 7.00) and similar between anatomic imaging modalities (CT: 1.09 to 3.56; MRI: 1.24 to 3.50). Inter-observer reproducibility was better for larger volumes. Perfused normal tissue volume RDC was 1.95 by anatomic and 3.19 by 99mTc-MAA SPECT, with corresponding absorbed dose RDC 1.46 and 1.75. Total perfused tumour volume RDC was higher, 2.92 for anatomic and 7.0 by 99mTc-MAA SPECT with corresponding absorbed dose RDC of 1.84 and 2.78. Intra-observer variability was lower for perfused NTAD (range: 14.3 to 19.7 Gy) than total perfused TAD (range: 42.8 to 121.4 Gy).ConclusionAnatomic segmentation-based dosimetry, versus 99mTc-MAA segmentation, results in lower absorbed doses with superior reproducibility. Higher volume compartments, such as normal tissue versus tumour, exhibit improved reproducibility.Trial registrationNCT03295006.

Project description:In previous work we have developed a fast sequence that focusses on cerebrospinal fluid (CSF) based on the long T2 of CSF. By processing the data obtained with this CSF MRI sequence, brain parenchymal volume (BPV) and intracranial volume (ICV) can be automatically obtained. The aim of this study was to assess the precision of the BPV and ICV measurements of the CSF MRI sequence and to validate the CSF MRI sequence by comparison with 3D T1-based brain segmentation methods.Ten healthy volunteers (2 females; median age 28 years) were scanned (3T MRI) twice with repositioning in between. The scan protocol consisted of a low resolution (LR) CSF sequence (0:57min), a high resolution (HR) CSF sequence (3:21min) and a 3D T1-weighted sequence (6:47min). Data of the HR 3D-T1-weighted images were downsampled to obtain LR T1-weighted images (reconstructed imaging time: 1:59 min). Data of the CSF MRI sequences was automatically segmented using in-house software. The 3D T1-weighted images were segmented using FSL (5.0), SPM12 and FreeSurfer (5.3.0).The mean absolute differences for BPV and ICV between the first and second scan for CSF LR (BPV/ICV: 12±9/7±4cc) and CSF HR (5±5/4±2cc) were comparable to FSL HR (9±11/19±23cc), FSL LR (7±4, 6±5cc), FreeSurfer HR (5±3/14±8cc), FreeSurfer LR (9±8, 12±10cc), and SPM HR (5±3/4±7cc), and SPM LR (5±4, 5±3cc). The correlation between the measured volumes of the CSF sequences and that measured by FSL, FreeSurfer and SPM HR and LR was very good (all Pearson's correlation coefficients >0.83, R2 .67-.97). The results from the downsampled data and the high-resolution data were similar.Both CSF MRI sequences have a precision comparable to, and a very good correlation with established 3D T1-based automated segmentations methods for the segmentation of BPV and ICV. However, the short imaging time of the fast CSF MRI sequence is superior to the 3D T1 sequence on which segmentation with established methods is performed.

Project description:The Genome Analysis Toolkit (GATK) is commonly used for variant calling of single nucleotide polymorphisms (SNPs) and small insertions and deletions (indels) from short-read sequencing data aligned against a reference genome. There have been a number of variant calling comparisons against GATK, but an equally comprehensive comparison for VarScan not yet been performed. More specifically, we compare (1) the effects of different pre-processing steps prior to variant calling with both GATK and VarScan, (2) VarScan variants called with increasingly conservative parameters, and (3) filtered and unfiltered GATK variant calls (for both the UnifiedGenotyper and the HaplotypeCaller). Variant calling was performed on three datasets (1 targeted exon dataset and 2 exome datasets), each with approximately a dozen subjects. In most cases, pre-processing steps (e.g., indel realignment and quality score base recalibration using GATK) had only a modest impact on the variant calls, but the importance of the pre-processing steps varied between datasets and variant callers. Based upon concordance statistics presented in this study, we recommend GATK users focus on "high-quality" GATK variants by filtering out variants flagged as low-quality. We also found that running VarScan with a conservative set of parameters (referred to as "VarScan-Cons") resulted in a reproducible list of variants, with high concordance (>97%) to high-quality variants called by the GATK UnifiedGenotyper and HaplotypeCaller. These conservative parameters result in decreased sensitivity, but the VarScan-Cons variant list could still recover 84-88% of the high-quality GATK SNPs in the exome datasets. This study also provides limited evidence that VarScan-Cons has a decreased false positive rate among novel variants (relative to high-quality GATK SNPs) and that the GATK HaplotypeCaller has an increased false positive rate for indels (relative to VarScan-Cons and high-quality GATK UnifiedGenotyper indels). More broadly, we believe the metrics used for comparison in this study can be useful in assessing the quality of variant calls in the context of a specific experimental design. As an example, a limited number of variant calling comparisons are also performed on two additional variant callers.

Dataset Information

Reproducibility and Bias in Healthy Brain Segmentation: Comparison of Two Popular Neuroimaging Platforms.

Publications

Reproducibility and Bias in Healthy Brain Segmentation: Comparison of Two Popular Neuroimaging Platforms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets