Dataset Information

Reproducibility of Segmentation-based Myocardial Radiomic Features with Cardiac MRI.

ABSTRACT:

Purpose

To investigate reproducibility of myocardial radiomic features with cardiac MRI.

Materials and methods

Test-retest studies were performed with a 3-T MRI system using commonly used cardiac MRI sequences of cine balanced steady-state free precession (cine bSSFP), T1-weighted and T2-weighted imaging, and quantitative T1 and T2 mapping in phantom experiments and 10 healthy participants (mean ± standard deviation age, 29 years ± 13). In addition, this study assessed repeatability in 51 patients (56 years ± 14) who underwent imaging twice during the same session. Three readers independently delineated the myocardium to investigate inter- and intraobserver reproducibility of radiomic features. A total of 1023 radiomic features were extracted by using PyRadiomics (https://pyradiomics.readthedocs.io/) with 11 image filters and six feature families. The intraclass correlation coefficient (ICC) was estimated to assess reproducibility and repeatability, and features with ICCs greater than or equal to 0.8 were considered reproducible.

Results

Different reproducibility patterns were observed among sequences in in vivo test-retest studies. In cine bSSFP, the gray-level run-length matrix was the most reproducible feature family, and the wavelet low-pass filter applied horizontally and vertically was the most reproducible image filter. In T1 and T2 maps, intensity-based statistics (first-order) and gray-level co-occurrence matrix features were the most reproducible feature families, without a dominant reproducible image filter. Across all sequences, gray-level nonuniformity was the most frequently identified reproducible feature name. In inter- and intraobserver reproducibility studies, respectively, only 32%-47% and 61%-73% of features were identified as reproducible.

Conclusion

Only a small subset of myocardial radiomic features was reproducible, and these reproducible radiomic features varied among different sequences. Supplemental material is available for this article. © RSNA, 2020See also the commentary by Leiner in this issue.

SUBMITTER: Jang J

PROVIDER: S-EPMC7377242 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:PurposeThe use of radiomic features as biomarkers of treatment response and outcome or as correlates to genomic variations requires that the computed features are robust and reproducible. Segmentation, a crucial step in radiomic analysis, is a major source of variability in the computed radiomic features. Therefore, we studied the impact of tumor segmentation variability on the robustness of MRI radiomic features.MethodFluid-attenuated inversion recovery (FLAIR) and contrast-enhanced T1-weighted (T1WICE ) MRI of 90 patients diagnosed with glioblastoma were segmented using a semiautomatic algorithm and an interactive segmentation with two different raters. We analyzed the robustness of 108 radiomic features from five categories (intensity histogram, gray-level co-occurrence matrix, gray-level size-zone matrix (GLSZM), edge maps, and shape) using intra-class correlation coefficient (ICC) and Bland and Altman analysis.ResultsOur results show that both segmentation methods are reliable with ICC ≥ 0.96 and standard deviation (SD) of mean differences between the two raters (SDdiffs ) ≤ 30%. Features computed from the histogram and co-occurrence matrices were found to be the most robust (ICC ≥ 0.8 and SDdiffs ≤ 30% for most features in these groups). Features from GLSZM were shown to have mixed robustness. Edge, shape, and GLSZM features were the most impacted by the choice of segmentation method with the interactive method resulting in more robust features than the semiautomatic method. Finally, features computed from T1WICE and FLAIR images were found to have similar robustness when computed with the interactive segmentation method.ConclusionSemiautomatic and interactive segmentation methods using two raters are both reliable. The interactive method produced more robust features than the semiautomatic method. We also found that the robustness of radiomic features varied by categories. Therefore, this study could help motivate segmentation methods and feature selection in MRI radiomic studies.

Project description:BackgroundCardiac magnetic resonance (MR) images are often collected with different imaging parameters, which may impact the calculated values of myocardial radiomic features.PurposeTo investigate the sensitivity of myocardial radiomic features to changes in imaging parameters in cardiac MR images.Study typeProspective.PopulationA total of 11 healthy participants/five patients.Field strength/ sequenceA 3 T/cine balanced steady-state free-precession, T1 -weighted spoiled gradient-echo, T2 -weighted turbo spin-echo, and quantitative T1 and T2 mapping. For each sequence, the flip angle, in-plane resolution, slice thickness, and parallel imaging technique were varied to study the sensitivity of radiomic features to alterations in imaging parameters.AssessmentMyocardial contours were manually delineated by experienced readers, and a total of 1023 radiomic features were extracted using PyRadiomics with 11 image filters and six feature families.Statistical testsSensitivity was defined as the standardized mean difference (D effect size), and the robust features were defined at sensitivity < 0.2. Sensitivity analysis was performed on predefined sets of reproducible features. The analysis was performed using the entire cohort of 16 subejcts.Results64% of radiomic features were robust (sensitivity < 0.2) to changes in any imaging parameter. In qualitative sequences, radiomic features were most sensitive to changes in in-plane spatial resolution (spatial resolution: 0.6 vs. flip angle: 0.19, parallel imaging: 0.18, slice thickness: 0.07; P < 0.01 for all); in quantitative sequences, radiomic features were least sensitive to changes in spatial resolution (spatial resolution: 0.07 vs. slice thickness: 0.16, flip angle: 0.24; P < 0.01 for all). In an individual feature level, no singular feature family/image filter was identified as robust (sensitivity < 0.2) across sequences; however, highly sensitive features were predominantly associated with high-frequency wavelet filters across all sequences (32/50 features).Data conclusionIn cardiac MR, a considerable number of radiomic features are sensitive to changes in sequence parameters.Evidence level1 TECHNICAL EFFICACY: Stage 1.

Project description:The use of multicentric data is becoming essential for developing generalizable radiomic signatures. In particular, Magnetic Resonance Imaging (MRI) data used in brain oncology are often heterogeneous in terms of scanners and acquisitions, which significantly impact quantitative radiomic features. Various methods have been proposed to decrease dependency, including methods acting directly on MR images, i.e., based on the application of several preprocessing steps before feature extraction or the ComBat method, which harmonizes radiomic features themselves. The ComBat method used for radiomics may be misleading and presents some limitations, such as the need to know the labels associated with the "batch effect". In addition, a statistically representative sample is required and the applicability of a signature whose batch label is not present in the train set is not possible. This work aimed to compare a priori and a posteriori radiomic harmonization methods and propose a code adaptation to be machine learning compatible. Furthermore, we have developed AutoComBat, which aims to automatically determine the batch labels, using either MRI metadata or quality metrics as inputs of the proposed constrained clustering. A heterogeneous dataset consisting of high and low-grade gliomas coming from eight different centers was considered. The different methods were compared based on their ability to decrease relative standard deviation of radiomic features extracted from white matter and on their performance on a classification task using different machine learning models. ComBat and AutoComBat using image-derived quality metrics as inputs for batch assignment and preprocessing methods presented promising results on white matter harmonization, but with no clear consensus for all MR images. Preprocessing showed the best results on the T1w-gd images for the grading task. For T2w-flair, AutoComBat, using either metadata plus quality metrics or metadata alone as inputs, performs better than the conventional ComBat, highlighting its potential for data harmonization. Our results are MRI weighting, feature class and task dependent and require further investigations on other datasets.

Project description:PurposeTo assess the performance of an automated myocardial T2 and extracellular volume (ECV) quantification method using transfer learning of a fully convolutional neural network (CNN) pretrained to segment the myocardium on T1 mapping images.Materials and methodsA single CNN previously trained and tested using 11 550 manually segmented native T1-weighted images was used to segment the myocardium for automated myocardial T2 and ECV quantification. Reference measurements from 1525 manually processed T2 maps and 1525 ECV maps (from 305 patients) were used to evaluate the performance of the pretrained network. Correlation coefficient (R) and Bland-Altman analysis were used to assess agreement between automated and reference values on per-patient, per-slice, and per-segment analyses. Furthermore, transfer learning effectiveness in the CNN was evaluated by comparing its performance to four CNNs trained using manually segmented T2-weighted and postcontrast T1-weighted images and initialized using random-weights or weights of the pretrained CNN.ResultsT2 and ECV measurements using the pretrained CNN strongly correlated with reference values in per-patient (T2: R = 0.88, 95% confidence interval [CI]: 0.85, 0.91; ECV: R = 0.91, 95% CI: 0.89, 0.93), per-slice (T2: R = 0.83, 95% CI: 0.81, 0.85; ECV: R = 0.84, 95% CI: 0.82, 0.86), and per-segment (T2: R = 0.75, 95% CI: 0.74, 0.77; ECV: R = 0.76, 95% CI: 0.75, 0.77) analyses. In Bland-Altman analysis, the automatic and reference values were in good agreement in per-patient (T2: 0.3 msec ± 2.9; ECV: -0.3% ± 1.7), per-slice (T2: 0.1 msec ± 4.6; ECV: -0.3% ± 2.5), and per-segment (T2: 0.0 msec ± 6.5; ECV: -0.4% ± 3.5) analyses. The performance of the pretrained network was comparable to networks refined or trained from scratch using additional manually segmented images.ConclusionTransfer learning extends the utility of pretrained CNN-based automated native T1 mapping analysis to T2 and ECV mapping without compromising performance. Supplemental material is available for this article. © RSNA, 2020.

Project description:Site-specific investigations of the role of radiomics in cancer diagnosis and therapy are emerging. We evaluated the reproducibility of radiomic features extracted from 18 Flourine-fluorodeoxyglucose (18 F-FDG) PET images for three parameters: manual versus computer-aided segmentation methods, gray-level discretization, and PET image reconstruction algorithms. Our cohort consisted of pretreatment PET/CT scans from 88 cervical cancer patients. Two board-certified radiation oncologists manually segmented the metabolic tumor volume (MTV1 and MTV2 ) for each patient. For comparison, we used a graphical-based method to generate semiautomated segmented volumes (GBSV). To address any perturbations in radiomic feature values, we down-sampled the tumor volumes into three gray-levels: 32, 64, and 128 from the original gray-level of 256. Finally, we analyzed the effect on radiomic features on PET images of eight patients due to four PET 3D-reconstruction algorithms: maximum likelihood-ordered subset expectation maximization (OSEM) iterative reconstruction (IR) method, fourier rebinning-ML-OSEM (FOREIR), FORE-filtered back projection (FOREFBP), and 3D-Reprojection (3DRP) analytical method. We extracted 79 features from all segmentation method, gray-levels of down-sampled volumes, and PET reconstruction algorithms. The features were extracted using gray-level co-occurrence matrices (GLCM), gray-level size zone matrices (GLSZM), gray-level run-length matrices (GLRLM), neighborhood gray-tone difference matrices (NGTDM), shape-based features (SF), and intensity histogram features (IHF). We computed the Dice coefficient between each MTV and GBSV to measure segmentation accuracy. Coefficient values close to one indicate high agreement, and values close to zero indicate low agreement. We evaluated the effect on radiomic features by calculating the mean percentage differences (d¯) between feature values measured from each pair of parameter elements (i.e. segmentation methods: MTV1 -MTV2 , MTV1 -GBSV, MTV2 -GBSV; gray-levels: 64-32, 64-128, and 64-256; reconstruction algorithms: OSEM-FORE-OSEM, OSEM-FOREFBP, and OSEM-3DRP). We used |d¯| as a measure of radiomic feature reproducibility level, where any feature scored |d¯| ±SD ? |25|% ± 35% was considered reproducible. We used Bland-Altman analysis to evaluate the mean, standard deviation (SD), and upper/lower reproducibility limits (U/LRL) for radiomic features in response to variation in each testing parameter. Furthermore, we proposed U/LRL as a method to classify the level of reproducibility: High- ±1% ? U/LRL ? ±30%; Intermediate- ±30% < U/LRL ? ±45%; Low- ±45 < U/LRL ? ±50%. We considered any feature below the low level as nonreproducible (NR). Finally, we calculated the interclass correlation coefficient (ICC) to evaluate the reliability of radiomic feature measurements for each parameter. The segmented volumes of 65 patients (81.3%) scored Dice coefficient >0.75 for all three volumes. The result outcomes revealed a tendency of higher radiomic feature reproducibility among segmentation pair MTV1 -GBSV than MTV2 -GBSV, gray-level pairs of 64-32 and 64-128 than 64-256, and reconstruction algorithm pairs of OSEM-FOREIR and OSEM-FOREFBP than OSEM-3DRP. Although the choice of cervical tumor segmentation method, gray-level value, and reconstruction algorithm may affect radiomic features, some features were characterized by high reproducibility through all testing parameters. The number of radiomic features that showed insensitivity to variations in segmentation methods, gray-level discretization, and reconstruction algorithms was 10 (13%), 4 (5%), and 1 (1%), respectively. These results suggest that a careful analysis of the effects of these parameters is essential prior to any radiomics clinical application.

Project description:PurposeAn ever-growing number of predictive models used to inform clinical decision making have included quantitative, computer-extracted imaging biomarkers, or "radiomic features." Broadly generalizable validity of radiomics-assisted models may be impeded by concerns about reproducibility. We offer a qualitative synthesis of 41 studies that specifically investigated the repeatability and reproducibility of radiomic features, derived from a systematic review of published peer-reviewed literature.Methods and materialsThe PubMed electronic database was searched using combinations of the broad Haynes and Ingui filters along with a set of text words specific to cancer, radiomics (including texture analyses), reproducibility, and repeatability. This review has been reported in compliance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. From each full-text article, information was extracted regarding cancer type, class of radiomic feature examined, reporting quality of key processing steps, and statistical metric used to segregate stable features.ResultsAmong 624 unique records, 41 full-text articles were subjected to review. The studies primarily addressed non-small cell lung cancer and oropharyngeal cancer. Only 7 studies addressed in detail every methodologic aspect related to image acquisition, preprocessing, and feature extraction. The repeatability and reproducibility of radiomic features are sensitive at various degrees to processing details such as image acquisition settings, image reconstruction algorithm, digital image preprocessing, and software used to extract radiomic features. First-order features were overall more reproducible than shape metrics and textural features. Entropy was consistently reported as one of the most stable first-order features. There was no emergent consensus regarding either shape metrics or textural features; however, coarseness and contrast appeared among the least reproducible.ConclusionsInvestigations of feature repeatability and reproducibility are currently limited to a small number of cancer types. Reporting quality could be improved regarding details of feature extraction software, digital image manipulation (preprocessing), and the cutoff value used to distinguish stable features.

Project description:BackgroundMyxoid tumors pose diagnostic challenges for radiologists and pathologists. All myxoid tumors can be differentiated from each other using fluorescent in-situ hybridization (FISH) or immunohistochemical markers, except for myxomas and myxofibrosarcomas. Myxomas and myxofibrosarcomas are rare tumors. Myxomas are benign and histologically bland, whereas myxofibrosarcomas are malignant and histologically heterogenous. Because of the histological heterogeneity, low grade myxofibrosarcomas may be mistaken for myxomas on core needle biopsies. We evaluated the performance of T1-weighted signal intensity (T1SI), tumor volume, and radiomic features extracted from magnetic resonance imaging (MRI) to differentiate myxomas from myxofibrosarcomas.MethodsThe MRIs of 56 patients (29 with myxomas, 27 with myxofibrosarcomas) were analyzed. We extracted 89 radiomic features. Random forests based classifiers using the T1SI, volume features, and radiomic features were used to differentiate myxomas from myxofibrosarcomas. The classifiers were validated using a leave-one-out cross-validation. The performances of the classifiers were then compared.ResultsMyxomas had lower normalized T1SI than myxofibrosaromas (p = 0.006) and the AUC using the T1SI was 0.713. However, the classification model using radiomic features had an AUC of 0.885 (accuracy = 0.839, sensitivity = 0.852, specificity = 0.828), and outperformed the classification models using T1SI (AUC = 0.713) and tumor volume (AUC = 0.838). The classification model using radiomic features was significantly better than the classifier using T1SI values (p = 0.039).ConclusionsMyxofibrosarcomas are on average higher in T1-weighted signal intensity than myxomas. Myxofibrosarcomas are larger and have shape differences compared to myxomas. Radiomic features performed best for differentiating myxomas from myxofibrosarcomas compared to T1-weighted signal intensity and tumor volume features.