Dataset Information

Inter- and Intra-Observer Repeatability of Quantitative Whole-Body, Diffusion-Weighted Imaging (WBDWI) in Metastatic Bone Disease.

ABSTRACT: Quantitative whole-body diffusion-weighted MRI (WB-DWI) is now possible using semi-automatic segmentation techniques. The method enables whole-body estimates of global Apparent Diffusion Coefficient (gADC) and total Diffusion Volume (tDV), both of which have demonstrated considerable utility for assessing treatment response in patients with bone metastases from primary prostate and breast cancers. Here we investigate the agreement (inter-observer repeatability) between two radiologists in their definition of Volumes Of Interest (VOIs) and subsequent assessment of tDV and gADC on an exploratory patient cohort of nine. Furthermore, each radiologist was asked to repeat his or her measurements on the same patient data sets one month later to identify the intra-observer repeatability of the technique. Using a Markov Chain Monte Carlo (MCMC) estimation method provided full posterior probabilities of repeatability measures along with maximum a-posteriori values and 95% confidence intervals. Our estimates of the inter-observer Intraclass Correlation Coefficient (ICCinter) for log-tDV and median gADC were 1.00 (0.97-1.00) and 0.99 (0.89-0.99) respectively, indicating excellent observer agreement for these metrics. Mean gADC values were found to have ICCinter = 0.97 (0.81-0.99) indicating a slight sensitivity to outliers in the derived distributions of gADC. Of the higher order gADC statistics, skewness was demonstrated to have good inter-user agreement with ICCinter = 0.99 (0.86-1.00), whereas gADC variance and kurtosis performed relatively poorly: 0.89 (0.39-0.97) and 0.96 (0.69-0.99) respectively. Estimates of intra-observer repeatability (ICCintra) demonstrated similar results: 0.99 (0.95-1.00) for log-tDV, 0.98 (0.89-0.99) and 0.97 (0.83-0.99) for median and mean gADC respectively, 0.64 (0.25-0.88) for gADC variance, 0.85 (0.57-0.95) for gADC skewness and 0.85 (0.57-0.95) for gADC kurtosis. Further investigation of two anomalous patient cases revealed that a very small proportion of voxels with outlying gADC values lead to instability in higher order gADC statistics. We therefore conclude that estimates of median/mean gADC and tumour volume demonstrate excellent inter- and intra-observer repeatability whilst higher order statistics of gADC should be used with caution when ascribing significance to clinical changes.

SUBMITTER: Blackledge MD

PROVIDER: S-EPMC4849763 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inter- and Intra-Observer Repeatability of Quantitative Whole-Body, Diffusion-Weighted Imaging (WBDWI) in Metastatic Bone Disease.

Blackledge Matthew D MD Tunariu Nina N Orton Matthew R MR Padhani Anwar R AR Collins David J DJ Leach Martin O MO Koh Dow-Mu DM

PloS one 20160428 4

Quantitative whole-body diffusion-weighted MRI (WB-DWI) is now possible using semi-automatic segmentation techniques. The method enables whole-body estimates of global Apparent Diffusion Coefficient (gADC) and total Diffusion Volume (tDV), both of which have demonstrated considerable utility for assessing treatment response in patients with bone metastases from primary prostate and breast cancers. Here we investigate the agreement (inter-observer repeatability) between two radiologists in their ...[more]

PMID: 27123931

Similar Datasets

Project description:The aim of the present study is to evaluate the intra- and inter-observer agreement in assessing the renal function by means of 99mTc-MAG3 diuretic renography. One hundred and twenty adults were enrolled in the study. One experienced and one junior radiographer processed the renograms twice by assigning manual and semi-automated regions of interest. The differential renal function (DRF, %), time to maximum counts for the right and left kidney (TmaxR-TmaxL, min) and time to half-peak counts (T1/2, min) were calculated. The Bland-Altman analysis (bias±95% limits of agreement), Lin's concordance correlation coefficient and weighted Fleiss' kappa coefficient were used to assess agreement. Based on the Bland-Altman analysis, the intra-observer repeatability results for the experienced radiographer using the manual and the semi-automated techniques were 0.2 ± 2.6% and 0.3 ± 6.4% (DRF), respectively, -0.01 ± 0.24 and 0.00 ± 0.34 (TmaxR), respectively, and 0.00 ± 0.26 and 0.00 ± 0.33 (TmaxL), respectively. For the junior radiographer, the respective results were 0.5 ± 5.0% and 0.8 ± 9.4% (DRF), 0.00 ± 0.44 and 0.01 ± 0.28 (TmaxR), and 0.01 ± 0.28 and -0.02 ± 0.44 (TmaxL). The inter-observer repeatability for the manual method was 0.6 ± 5.0% (DRF), -0.10 ± 0.42 (TmaxR) and -0.05 ± 0.38 (TmaxL), and for the semi-automated method -0.2 ± 9.1% (DRF), 0.00 ± 0.31 (TmaxR) and -0.05 ± 0.40 (TmaxL). The weighted Fleiss' kappa coefficient for the T1/2 assessments ranged between 0.85-0.97 for both intra- and inter-observer repeatability with both methods. These findings suggest a very good repeatability in DRF assessment with the manual method-especially for the experienced observer-but a less good repeatability with the semi-automated approach. The calculation of Tmax was also operator-dependent. We conclude that reader experience is important in the calculation of renal parameters. We therefore encourage reader training in renal scintigraphy. Moreover, the manual tool seems to perform better than the semi-automated tool. Thus, we encourage cautious use of automated tools and adjunct validation by manual methods where possible.

Project description:BackgroundWhole-Body Diffusion-Weighted Imaging (WBDWI) is an established technique for staging and evaluating treatment response in patients with multiple myeloma (MM) and advanced prostate cancer (APC). However, WBDWI scans show inter- and intra-patient intensity signal variability. This variability poses challenges in accurately quantifying bone disease, tracking changes over follow-up scans, and developing automated tools for bone lesion delineation. Here, we propose a novel automated pipeline for inter-station, inter-scan image signal standardisation on WBDWI that utilizes robust segmentation of the spinal canal through deep learning.MethodsWe trained and validated a supervised 2D U-Net model to automatically delineate the spinal canal (both the spinal cord and surrounding cerebrospinal fluid, CSF) in an initial cohort of 40 patients who underwent WBDWI for treatment response evaluation (80 scans in total). Expert-validated contours were used as the target standard. The algorithm was further semi-quantitatively validated on four additional datasets (three internal, one external, 207 scans total) by comparing the distributions of average apparent diffusion coefficient (ADC) and volume of the spinal cord derived from a two-component Gaussian mixture model of segmented regions. Our pipeline subsequently standardises WBDWI signal intensity through two stages: (i) normalisation of signal between imaging stations within each patient through histogram equalisation of slices acquired on either side of the station gap, and (ii) inter-scan normalisation through histogram equalisation of the signal derived within segmented spinal canal regions. This approach was semi-quantitatively validated in all scans available to the study (N = 287).ResultsThe test dice score, precision, and recall of the spinal canal segmentation model were all above 0.87 when compared to manual delineation. The average ADC for the spinal cord (1.7 × 10-3 mm2/s) showed no significant difference from the manual contours. Furthermore, no significant differences were found between the average ADC values of the spinal cord across the additional four datasets. The signal-normalised, high-b-value images were visualised using a fixed contrast window level and demonstrated qualitatively better signal homogeneity across scans than scans that were not signal-normalised.ConclusionOur proposed intensity signal WBDWI normalisation pipeline successfully harmonises intensity values across multi-centre cohorts. The computational time required is less than 10 s, preserving contrast-to-noise and signal-to-noise ratios in axial diffusion-weighted images. Importantly, no changes to the clinical MRI protocol are expected, and there is no need for additional reference MRI data or follow-up scans.

Project description:ObjectiveTo assess the test-retest reproducibility and intra/interobserver agreement of apparent diffusion coefficient (ADC) measurements of myeloma lesions using whole body diffusion-weighted MRI (WB-DW-MRI) at 3T MRI.MethodsFollowing ethical approval, 11 consenting patients with relapsed multiple myeloma were prospectively recruited and underwent baseline WB-DW-MRI. For a single bed position, axial DWI was repeated after a short interval to permit test-retest measurements.Mean ADC measurement was performed by two experienced observers. Intra- and interobserver agreement and test-retest reproducibility were assessed, using coefficient of variation (CV) and interclass correlation coefficient (ICC) measures, for diffuse and focal lesions (small ≤10 mm and large >10 mm).Results47 sites of disease were outlined (23 focal, 24 diffuse) in different bed positions (pelvis = 22, thorax = 20, head and neck = 5). For all lesions, there was excellent intraobserver agreement with ICC of 0.99 (0.98-0.99) and COV of 5%. For interobserver agreement, ICC was 0.89 (0.8-0.934) and COV was 17%. There was poor interobserver agreement for diffuse disease (ICC = 0.46) and small lesions (ICC = 0.54).For test-retest reproducibility, excellent ICC (0.916) and COV (14.5%) values for mean ADC measurements were observed. ICCs of test-retest were similar between focal lesions (0.83) and diffuse infiltration (0.80), while ICCs were higher in pelvic (0.95) compared to thoracic (0.81) region and in small (0.96) compared to large (0.8) lesions.ConclusionADC measurements of focal lesions in multiple myeloma are repeatable and reproducible, while there is more variation in ADC measurements of diffuse disease in patients with multiple myeloma.Advances in knowledgeMean ADC measurements are repeatable and reproducible in focal lesions in multiple myeloma, while the ADC measurements of diffuse disease in multiple myeloma are more subject to variation. The evidence supports the future potential role of ADC measurements as predictive quantitative biomarker in multiple myeloma.

Dataset Information

Inter- and Intra-Observer Repeatability of Quantitative Whole-Body, Diffusion-Weighted Imaging (WBDWI) in Metastatic Bone Disease.

Publications

Inter- and Intra-Observer Repeatability of Quantitative Whole-Body, Diffusion-Weighted Imaging (WBDWI) in Metastatic Bone Disease.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets