Dataset Information

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.

ABSTRACT: OBJECTIVE:To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome. METHODS:The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung cancer). Features were calculated using four platforms (PyRadiomics, LIFEx, CERR and IBEX). PyRadiomics, LIFEx and CERR are IBSI-compliant, whereas IBEX is not. The effects of IBSI compliance, user-defined calculation settings and platform version were assessed by calculating intraclass correlation coefficients and confidence intervals. The influence of platform choice on the relationship between radiomic biomarkers and survival was evaluated using univariable cox regression in the largest dataset. RESULTS:The reliability of radiomic features calculated by the different software platforms was only excellent (ICC > 0.9) for 4/17 radiomic features when comparing all four platforms. Reliability improved to ICC > 0.9 for 15/17 radiomic features when analysis was restricted to the three IBSI-compliant platforms. Failure to harmonise calculation settings resulted in poor reliability, even across the IBSI-compliant platforms. Software platform version also had a marked effect on feature reliability in CERR and LIFEx. Features identified as having significant relationship to survival varied between platforms, as did the direction of hazard ratios. CONCLUSION:IBSI compliance, user-defined calculation settings and choice of platform version all influence the statistical reliability and corresponding performance of prognostic models in radiomics. KEY POINTS:• Reliability of radiomic features varies between feature calculation platforms and with choice of software version. • Image Biomarker Standardisation Initiative (IBSI) compliance improves reliability of radiomic features across platforms, but only when calculation settings are harmonised. • IBSI compliance, user-defined calculation settings and choice of platform version collectively affect the prognostic value of features.

SUBMITTER: Fornacon-Wood I

PROVIDER: S-EPMC7553896 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.

Fornacon-Wood Isabella I Mistry Hitesh H Ackermann Christoph J CJ Blackhall Fiona F McPartlin Andrew A Faivre-Finn Corinne C Price Gareth J GJ O'Connor James P B JPB

European radiology 20200601 11

<h4>Objective</h4>To investigate the effects of Image Biomarker Standardisation Initiative (IBSI) compliance, harmonisation of calculation settings and platform version on the statistical reliability of radiomic features and their corresponding ability to predict clinical outcome.<h4>Methods</h4>The statistical reliability of radiomic features was assessed retrospectively in three clinical datasets (patient numbers: 108 head and neck cancer, 37 small-cell lung cancer, 47 non-small-cell lung canc ...[more]

PMID: 32483644

Similar Datasets

Project description:PurposeFor optimal management of ductal carcinoma in situ (DCIS), reproducible histopathological assessment is essential to distinguish low-risk from high-risk DCIS. Therefore, we analyzed interrater reliability of histopathological DCIS features and assessed their associations with subsequent ipsilateral invasive breast cancer (iIBC) risk.MethodsUsing a case-cohort design, reliability was assessed in a population-based, nationwide cohort of 2767 women with screen-detected DCIS diagnosed between 1993 and 2004, treated by breast-conserving surgery with/without radiotherapy (BCS ± RT) using Krippendorff's alpha (KA) and Gwet's AC2 (GAC2). Thirty-eight raters scored histopathological DCIS features including grade (2-tiered and 3-tiered), growth pattern, mitotic activity, periductal fibrosis, and lymphocytic infiltrate in 342 women. Using majority opinion-based scores for each feature, their association with subsequent iIBC risk was assessed using Cox regression.ResultsInterrater reliability of grade using various classifications was fair to moderate, and only substantial for grade 1 versus 2 + 3 when using GAC2 (0.78). Reliability for growth pattern (KA 0.44, GAC2 0.78), calcifications (KA 0.49, GAC2 0.70) and necrosis (KA 0.47, GAC2 0.70) was moderate using KA and substantial using GAC2; for (type of) periductal fibrosis and lymphocytic infiltrate fair to moderate estimates were found and for mitotic activity reliability was substantial using GAC2 (0.70). Only in patients treated with BCS-RT, high mitotic activity was associated with a higher iIBC risk in univariable analysis (Hazard Ratio (HR) 2.53, 95% Confidence Interval (95% CI) 1.05-6.11); grade 3 versus 1 + 2 (HR 2.64, 95% CI 1.35-5.14) and a cribriform/solid versus flat epithelial atypia/clinging/(micro)papillary growth pattern (HR 3.70, 95% CI 1.34-10.23) were independently associated with a higher iIBC risk.ConclusionsUsing majority opinion-based scores, DCIS grade, growth pattern, and mitotic activity are associated with iIBC risk in patients treated with BCS-RT, but interrater variability is substantial. Semi-quantitative grading, incorporating and separately evaluating nuclear pleomorphism, growth pattern, and mitotic activity, may improve the reliability and prognostic value of these features.

Project description:BACKGROUND:Textural features extracted from MRI potentially provide prognostic information additional to volume for influencing surgical management of cervical cancer. PURPOSE:To identify textural features that differ between cervical tumors above and below the volume threshold of eligibility for trachelectomy and determine their value in predicting recurrence in patients with low-volume tumors. METHODS:Of 378 patients with Stage1-2 cervical cancer imaged prospectively (3T, endovaginal coil), 125 had well-defined, histologically-confirmed squamous or adenocarcinomas with >100 voxels (>0.07 cm3) suitable for radiomic analysis. Regions-of-interest outlined the whole tumor on T2-W images and apparent diffusion coefficient (ADC) maps. Textural features based on grey-level co-occurrence matrices were compared (Mann-Whitney test with Bonferroni correction) between tumors greater (n = 46) or less (n = 79) than 4.19 cm3. Clustering eliminated correlated variables. Significantly different features were used to predict recurrence (regression modelling) in surgically-treated patients with low-volume tumors and compared with a model using clinico-pathological features. RESULTS:Textural features (Dissimilarity, Energy, ClusterProminence, ClusterShade, InverseVariance, Autocorrelation) in 6 of 10 clusters from T2-W and ADC data differed between high-volume (mean ± SD 15.3 ± 11.7 cm3) and low-volume (mean ± SD 1.3 ± 1.2 cm3) tumors. (p < 0.02). In low-volume tumors, predicting recurrence was indicated by: Dissimilarity, Energy (ADC-radiomics, AUC = 0.864); Dissimilarity, ClusterProminence, InverseVariance (T2-W-radiomics, AUC = 0.808); Volume, Depth of Invasion, LymphoVascular Space Invasion (clinico-pathological features, AUC = 0.794). Combining ADC-radiomic (but not T2-radiomic) and clinico-pathological features improved prediction of recurrence compared to the clinico-pathological model (AUC = 0.916, p = 0.006). Findings were supported by bootstrap re-sampling (n = 1000). CONCLUSION:Textural features from ADC maps and T2-W images differ between high- and low-volume tumors and potentially predict recurrence in low-volume tumors.

Project description:Background Using high robust radiomic features in modeling is recommended, yet its impact on radiomic model is unclear. This study evaluated the radiomic model’s robustness and generalizability after screening out low-robust features before radiomic modeling. The results were validated with four datasets and two clinically relevant tasks. Materials and methods A total of 1,419 head-and-neck cancer patients’ computed tomography images, gross tumor volume segmentation, and clinically relevant outcomes (distant metastasis and local-regional recurrence) were collected from four publicly available datasets. The perturbation method was implemented to simulate images, and the radiomic feature robustness was quantified using intra-class correlation of coefficient (ICC). Three radiomic models were built using all features (ICC > 0), good-robust features (ICC > 0.75), and excellent-robust features (ICC > 0.95), respectively. A filter-based feature selection and Ridge classification method were used to construct the radiomic models. Model performance was assessed with both robustness and generalizability. The robustness of the model was evaluated by the ICC, and the generalizability of the model was quantified by the train-test difference of Area Under the Receiver Operating Characteristic Curve (AUC). Results The average model robustness ICC improved significantly from 0.65 to 0.78 (P< 0.0001) using good-robust features and to 0.91 (P< 0.0001) using excellent-robust features. Model generalizability also showed a substantial increase, as a closer gap between training and testing AUC was observed where the mean train-test AUC difference was reduced from 0.21 to 0.18 (P< 0.001) in good-robust features and to 0.12 (P< 0.0001) in excellent-robust features. Furthermore, good-robust features yielded the best average AUC in the unseen datasets of 0.58 (P< 0.001) over four datasets and clinical outcomes. Conclusions Including robust only features in radiomic modeling significantly improves model robustness and generalizability in unseen datasets. Yet, the robustness of radiomic model has to be verified despite building with robust radiomic features, and tightly restricted feature robustness may prevent the optimal model performance in the unseen dataset as it may lower the discrimination power of the model.

Project description:PurposeThe use of radiomic features as biomarkers of treatment response and outcome or as correlates to genomic variations requires that the computed features are robust and reproducible. Segmentation, a crucial step in radiomic analysis, is a major source of variability in the computed radiomic features. Therefore, we studied the impact of tumor segmentation variability on the robustness of MRI radiomic features.MethodFluid-attenuated inversion recovery (FLAIR) and contrast-enhanced T1-weighted (T1WICE ) MRI of 90 patients diagnosed with glioblastoma were segmented using a semiautomatic algorithm and an interactive segmentation with two different raters. We analyzed the robustness of 108 radiomic features from five categories (intensity histogram, gray-level co-occurrence matrix, gray-level size-zone matrix (GLSZM), edge maps, and shape) using intra-class correlation coefficient (ICC) and Bland and Altman analysis.ResultsOur results show that both segmentation methods are reliable with ICC ≥ 0.96 and standard deviation (SD) of mean differences between the two raters (SDdiffs ) ≤ 30%. Features computed from the histogram and co-occurrence matrices were found to be the most robust (ICC ≥ 0.8 and SDdiffs ≤ 30% for most features in these groups). Features from GLSZM were shown to have mixed robustness. Edge, shape, and GLSZM features were the most impacted by the choice of segmentation method with the interactive method resulting in more robust features than the semiautomatic method. Finally, features computed from T1WICE and FLAIR images were found to have similar robustness when computed with the interactive segmentation method.ConclusionSemiautomatic and interactive segmentation methods using two raters are both reliable. The interactive method produced more robust features than the semiautomatic method. We also found that the robustness of radiomic features varied by categories. Therefore, this study could help motivate segmentation methods and feature selection in MRI radiomic studies.

Dataset Information

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.

Publications

Reliability and prognostic value of radiomic features are highly dependent on choice of feature extraction platform.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets