Dataset Information

Inter-observer and inter-examination variability of manual vertebral bone attenuation measurements on computed tomography.

ABSTRACT:

Objective

To determine inter-observer and inter-examination variability of manual attenuation measurements of the vertebrae in low-dose unenhanced chest computed tomography (CT).

Methods

Three hundred and sixty-seven lung cancer screening trial participants who underwent baseline and repeat unenhanced low-dose CT after 3 months because of an indeterminate lung nodule were included. The CT attenuation value of the first lumbar vertebrae (L1) was measured in all CTs by one observer to obtain inter-examination reliability. Six observers performed measurements in 100 randomly selected CTs to determine agreement with limits of agreement and Bland-Altman plots and reliability with intraclass correlation coefficients (ICCs). Reclassification analyses were performed using a threshold of 110 HU to define osteoporosis.

Results

Inter-examination reliability was excellent with an ICC of 0.92 (p < 0.001). Inter-examination limits of agreement ranged from -26 to 28 HU with a mean difference of 1 ± 14 HU. Inter-observer reliability ICCs ranged from 0.70 to 0.91. Inter-examination variability led to 11.2 % reclassification of participants and inter-observer variability led to 22.1 % reclassification.

Conclusions

Vertebral attenuation values can be manually quantified with good to excellent inter-examination and inter-observer reliability on unenhanced low-dose chest CT. This information is valuable for early detection of osteoporosis on low-dose chest CT.

Key points

• Vertebral attenuation values can be manually quantified on low-dose unenhanced CT reliably. • Vertebral attenuation measurements may be helpful in detecting subclinical low bone density. • This could become of importance in the detection of osteoporosis.

SUBMITTER: Pompe E

PROVIDER: S-EPMC4972882 | biostudies-literature | 2016 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inter-observer and inter-examination variability of manual vertebral bone attenuation measurements on computed tomography.

Pompe Esther E de Jong Pim A PA de Jong Werner U WU Takx Richard A P RA Eikendal Anouk L M AL Willemink Martin J MJ Oudkerk Matthijs M Budde Ricardo P J RP Lammers Jan-Willem J JW Mohamed Hoesein Firdaus A A FA

European radiology 20160122 9

<h4>Objective</h4>To determine inter-observer and inter-examination variability of manual attenuation measurements of the vertebrae in low-dose unenhanced chest computed tomography (CT).<h4>Methods</h4>Three hundred and sixty-seven lung cancer screening trial participants who underwent baseline and repeat unenhanced low-dose CT after 3 months because of an indeterminate lung nodule were included. The CT attenuation value of the first lumbar vertebrae (L1) was measured in all CTs by one observer ...[more]

PMID: 26801161

Similar Datasets

Project description:BackgroundA growing number of research studies have reported inter-observer variability in sizes of tumours measured from CT scans. It remains unclear whether the conventional statistical measures correctly evaluate the CT measurement consistency for optimal treatment management and decision-making. We compared and evaluated the existing measures for evaluating inter-observer variability in CT measurement of cancer lesions.Methods13 board-certified radiologists repeatedly reviewed 10 CT image sets of lung lesions and hepatic metastases selected through a randomisation process. A total of 130 measurements under RECIST 1.1 (Response Evaluation Criteria in Solid Tumors) guidelines were collected for the demonstration. Intraclass correlation coefficient (ICC), Bland-Altman plotting and outlier counting methods were selected for the comparison. The each selected measure was used to evaluate three cases with observed, increased and decreased inter-observer variability.ResultsThe ICC score yielded a weak detection when evaluating different levels of the inter-observer variability among radiologists (increased: 0.912; observed: 0.962; decreased: 0.990). The outlier counting method using Bland-Altman plotting with 2SD yielded no detection at all with its number of outliers unchanging regardless of level of inter-observer variability. Outlier counting based on domain knowledge was more sensitised to different levels of the inter-observer variability compared with the conventional measures (increased: 0.756; observed: 0.923; improved: 1.000). Visualisation of pairwise Bland-Altman bias was also sensitised to the inter-observer variability with its pattern rapidly changing in response to different levels of the inter-observer variability.ConclusionsConventional measures may yield weak or no detection when evaluating different levels of the inter-observer variability among radiologists. We observed that the outlier counting based on domain knowledge was sensitised to the inter-observer variability in CT measurement of cancer lesions. Our study demonstrated that, under certain circumstances, the use of standard statistical correlation coefficients may be misleading and result in a sense of false security related to the consistency of measurement for optimal treatment management and decision-making.

Project description:ObjectivesTo investigate the interdisciplinary interobserver reproducibility of Hertel-exophthalmometry-like protrusion measurements on multidetector-row-computed-tomography- (MDCT-) images of the orbit to facilitate structured evaluation of the orbit and mid-face.MethodsRespective reproducibility of base-length along the interfronto-zygomatic line, right and left ocular protrusion, and deriving interocular difference was measured in this retrospective (04/2009-03/2020) single-centre observational study. MDCT-series and slice-positions were selected independently, using picture-archiving-and-communication-system- (PACS-) tools on tilt-corrected axial MDCT-images (slice-thickness 0.6-3.0 mm, window/centre 350/50 HU) in 37 selected adult patients (24 female, age 57 ± 13 years, average±standard-deviation) with clinical indication for Hertel-exophthalmometry, by one radiology-attending, two ophthalmology-attendings, one critical-care-attending, and one ear-nose-throat-surgery resident, respectively. Bland-Altman plots and Wilcoxon-matched-pairs-signed-rank-tests compared interobserver results.ResultsMean and median interobserver and intraobserver (radiology-attending) deviations were within 1 mm of respective averages of base-length (98 ± 4 mm), right and left ocular protrusion (21 ± 4 mm) and interocular difference (2 ± 1 mm). Relative interobserver deviations were within 2.0% of average (all patients) for base-length, and 5.0% (>80% of patients) for ocular protrusion. Pairwise interobserver comparison showed no significant differences between interocular differences of protrusion.ConclusionsRespective measurements of base-length, ocular protrusion, and deriving interocular difference show high interdisciplinary interobserver reproducibility in tilt-corrected axial MDCT-images of the orbit or mid-face.Advances in knowledgeHertel-exophthalmometry-like protrusion measurements did not depend on the years of experience or the medical subspecialty of the observer. Measurements are objective, well reproducible and important for multiple medical disciplines and should thus be included in pertinent radiology reports.

Project description:PurposeCervical cancer metabolic tumour volume (MTV) derived from [18F]-FDG PET/CT has a role in prognostication and therapy planning. There is no standard method of outlining MTV on [18F]-FDG PET/CT. The aim of this study was to assess the optimal method to outline primary cervical tumours on [18F]-FDG PET/CT using MRI-derived tumour volumes as the reference standard.Methods81 consecutive cervical cancer patients with pre-treatment staging MRI and [18F]-FDG PET/CT imaging were included. MRI volumes were compared with different PET segmentation methods. Method 1 measured MTVs at different SUVmax thresholds ranging from 20 to 60% (MTV20-MTV60) with bladder masking and manual adjustment when required. Method 2 created an isocontour around the tumour prior to different SUVmax thresholds being applied. Method 3 used an automated gradient method. Inter-observer agreement of MTV, following manual adjustment when required, was recorded.ResultsFor method 1, the MTV25 and MTV30 were closest to the MRI volumes for both readers (mean percentage change from MRI volume of 2.9% and 13.4% for MTV25 and - 13.1% and - 2.0% for MTV30 for readers 1 and 2). 70% of lesions required manual adjustment at MTV25 compared with 45% at MTV30. There was excellent inter-observer agreement between MTV30 to MTV60 (ICC ranged from 0.898-0.976 with narrow 95% confidence intervals (CIs)) and moderate agreement at lower thresholds (ICC estimates of 0.534 and 0.617, respectively for the MTV20 and MTV25 with wide 95% CIs). Bladder masking was performed in 86% of cases overall. For method 2, excellent correlation was demonstrated at MTV25 and MTV30 (mean % change from MRI volume of -3.9% and - 8.6% for MTV25 and - 16.9% and 19% for MTV30 for readers 1 and 2, respectively). This method also demonstrated excellent ICC across all thresholds with no manual adjustment. Method 3 demonstrated excellent ICC of 0.96 (95% CI 0.94-0.97) but had a mean percentage difference from the MRI volume of - 19.1 and - 18.2% for readers 1 and 2, respectively. 21% required manual adjustment for both readers.ConclusionMTV30 provides the optimal correlation with MRI volume taking into consideration the excellent inter-reader agreement and less requirement for manual adjustment.

Project description:ObjectivesEstablishing the reproducibility of expert-derived measurements on CTA exams of aortic dissection is clinically important and paramount for ground-truth determination for machine learning.MethodsFour independent observers retrospectively evaluated CTA exams of 72 patients with uncomplicated Stanford type B aortic dissection and assessed the reproducibility of a recently proposed combination of four morphologic risk predictors (maximum aortic diameter, false lumen circumferential angle, false lumen outflow, and intercostal arteries). For the first inter-observer variability assessment, 47 CTA scans from one aortic center were evaluated by expert-observer 1 in an unconstrained clinical assessment without a standardized workflow and compared to a composite of three expert-observers (observers 2-4) using a standardized workflow. A second inter-observer variability assessment on 30 out of the 47 CTA scans compared observers 3 and 4 with a constrained, standardized workflow. A third inter-observer variability assessment was done after specialized training and tested between observers 3 and 4 in an external population of 25 CTA scans. Inter-observer agreement was assessed with intraclass correlation coefficients (ICCs) and Bland-Altman plots.ResultsPre-training ICCs of the four morphologic features ranged from 0.04 (-0.05 to 0.13) to 0.68 (0.49-0.81) between observer 1 and observers 2-4 and from 0.50 (0.32-0.69) to 0.89 (0.78-0.95) between observers 3 and 4. ICCs improved after training ranging from 0.69 (0.52-0.87) to 0.97 (0.94-0.99), and Bland-Altman analysis showed decreased bias and limits of agreement.ConclusionsManual morphologic feature measurements on CTA images can be optimized resulting in improved inter-observer reliability. This is essential for robust ground-truth determination for machine learning models.Key points• Clinical fashion manual measurements of aortic CTA imaging features showed poor inter-observer reproducibility. • A standardized workflow with standardized training resulted in substantial improvements with excellent inter-observer reproducibility. • Robust ground truth labels obtained manually with excellent inter-observer reproducibility are key to develop reliable machine learning models.

Dataset Information

Inter-observer and inter-examination variability of manual vertebral bone attenuation measurements on computed tomography.

Objective

Methods

Results

Conclusions

Key points

Publications

Inter-observer and inter-examination variability of manual vertebral bone attenuation measurements on computed tomography.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets