Dataset Information

Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain.

ABSTRACT: The purpose of this study was to investigate the correlation between model observer and human observer performance in CT imaging for the task of lesion detection and localization when the lesion location is uncertain.Two cylindrical rods (3-mm and 5-mm diameters) were placed in a 35×26 cm torso-shaped water phantom to simulate lesions with -15 HU contrast at 120 kV. The phantom was scanned 100 times on a 128-slice CT scanner at each of four dose levels (CTDIvol=5.7, 11.4, 17.1, and 22.8 mGy). Regions of interest (ROIs) around each lesion were extracted to generate images with signal-present, with each ROI containing 128×128 pixels. Corresponding ROIs of signal-absent images were generated from images without lesion mimicking rods. The location of the lesion (rod) in each ROI was randomly distributed by moving the ROIs around each lesion. Human observer studies were performed by having three trained observers identify the presence or absence of lesions, indicating the lesion location in each image and scoring confidence for the detection task on a 6-point scale. The same image data were analyzed using a channelized Hotelling model observer (CHO) with Gabor channels. Internal noise was added to the decision variables for the model observer study. Area under the curve (AUC) of ROC and localization ROC (LROC) curves were calculated using a nonparametric approach. The Spearman's rank order correlation between the average performance of the human observers and the model observer performance was calculated for the AUC of both ROC and LROC curves for both the 3- and 5-mm diameter lesions.In both ROC and LROC analyses, AUC values for the model observer agreed well with the average values across the three human observers. The Spearman's rank order correlation values for both ROC and LROC analyses for both the 3- and 5-mm diameter lesions were all 1.0, indicating perfect rank ordering agreement of the figures of merit (AUC) between the average performance of the human observers and the model observer performance.In CT imaging of different sizes of low-contrast lesions (-15 HU), the performance of CHO with Gabor channels was highly correlated with human observer performance for the detection and localization tasks with uncertain lesion location in CT imaging at four clinically relevant dose levels. This suggests the ability of Gabor CHO model observers to meaningfully assess CT image quality for the purpose of optimizing scan protocols and radiation dose levels in detection and localization tasks for low-contrast lesions.

SUBMITTER: Leng S

PROVIDER: S-EPMC3724792 | biostudies-other | 2013 Aug

REPOSITORIES: biostudies-other

ACCESS DATA

Similar Datasets

Project description:Model observers have been successfully developed and used to assess the quality of static 2D CT images. However, radiologists typically read images by paging through multiple 2D slices (i.e., multislice reading). The purpose of this study was to correlate human and model observer performance in a low-contrast detection task performed using both 2D and multislice reading, and to determine if the 2D model observer still correlate well with human observer performance in multislice reading.A phantom containing 18 low-contrast spheres (6 sizes × 3 contrast levels) was scanned on a 192-slice CT scanner at five dose levels (CTDIvol = 27, 13.5, 6.8, 3.4, and 1.7 mGy), each repeated 100 times. Images were reconstructed using both filtered-backprojection (FBP) and an iterative reconstruction (IR) method (ADMIRE, Siemens). A 3D volume of interest (VOI) around each sphere was extracted and placed side-by-side with a signal-absent VOI to create a 2-alternative forced choice (2AFC) trial. Sixteen 2AFC studies were generated, each with 100 trials, to evaluate the impact of radiation dose, lesion size and contrast, and reconstruction methods on object detection. In total, 1600 trials were presented to both model and human observers. Three medical physicists acted as human observers and were allowed to page through the 3D volumes to make a decision for each 2AFC trial. The human observer performance was compared with the performance of a multislice channelized Hotelling observer (CHO_MS), which integrates multislice image data, and with the performance of previously validated CHO, which operates on static 2D images (CHO_2D). For comparison, the same 16 2AFC studies were also performed in a 2D viewing mode by the human observers and compared with the multislice viewing performance and the two CHO models.Human observer performance was well correlated with the CHO_2D performance in the 2D viewing mode [Pearson product-moment correlation coefficient R = 0.972, 95% confidence interval (CI): 0.919 to 0.990] and with the CHO_MS performance in the multislice viewing mode (R = 0.952, 95% CI: 0.865 to 0.984). The CHO_2D performance, calculated from the 2D viewing mode, also had a strong correlation with human observer performance in the multislice viewing mode (R = 0.957, 95% CI: 879 to 0.985). Human observer performance varied between the multislice and 2D modes. One reader performed better in the multislice mode (P = 0.013); whereas the other two readers showed no significant difference between the two viewing modes (P = 0.057 and P = 0.38).A 2D CHO model is highly correlated with human observer performance in detecting spherical low contrast objects in multislice viewing of CT images. This finding provides some evidence for the use of a simpler, 2D CHO to assess image quality in clinically relevant CT tasks where multislice viewing is used.

Project description:A human observer study was performed for a signal detection task for the case of fan-beam x-ray computed tomography. Hotelling observer (HO) performance was calculated for the same detection task without the use of efficient channels. By considering the full image covariance produced by the filtered backprojection (FBP) algorithm and avoiding the use of channels in the computation of HO performance, the authors establish an absolute upper bound on signal detectability. Therefore, this study serves as a baseline for relating human and ideal observer performance in the case of fan-beam CT.Eight human observers participated in a two-alternative forced choice experiment where the signal of interest was a small simulated ellipsoid in the presence of independent, identically distributed Gaussian detector noise. Theoretical performance of the HO, which is equivalent to the ideal observer in this case (see Sec. 13.2.12 in Barrett and Myers [Foundations of Image Science (Wiley, Hoboken, NJ, 2004)], was also computed and compared to the performance of the human observers. In addition to a reference FBP implementation, two FBP implementations with inherent loss of HO signal detectability (e.g., by apodizing the ramp filter) were also investigated. Each of these latter two implementations takes the form of a discrete-to-discrete linear operator (i.e., a matrix), which has a nontrivial null-space resulting in the loss of detectability.Estimated observer detectability index (d(A)) values for the human observers and SNR values for the HO were obtained. While Hanning filtering in the FBP implementation with a cutoff frequency of 1/4 of the Nyquist frequency reduces HO SNR (due to the reconstruction matrix's nontrivial null-space), this filtering was shown to consistently improve human observer performance. By contrast, increasing the image pixel size was seen to have a comparable effect on both the HO and the human observers, degrading performance.These results, which characterize HO and human observer performance for a signal detection task in fan-beam FBP noise, form a basis for applying model observer metrics to fan-beam CT when knowledge of the full image-domain noise statistics is important. Further, by calculating HO performance without relying on channels, these results are particularly relevant when an information theoretic approach is considered, e.g., in optimization of the image reconstruction algorithm with respect to preservation of signal detectability. Finally, the HO (which is here equivalent to the ideal observer) provides an absolute upper bound on detection performance, and our results therefore provide insight into the performance of human observers relative to the optimum for two different cases wherein ideal observer performance is compromised through degradation of the data quality. In one case (regularization), human performance is improved to practically ideal performance, and in the other (larger pixel size), ideal and human observer performance are approximately degraded equivalently.

Dataset Information

Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets