Dataset Information

Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT.

ABSTRACT: Model observers have been successfully developed and used to assess the quality of static 2D CT images. However, radiologists typically read images by paging through multiple 2D slices (i.e., multislice reading). The purpose of this study was to correlate human and model observer performance in a low-contrast detection task performed using both 2D and multislice reading, and to determine if the 2D model observer still correlate well with human observer performance in multislice reading.A phantom containing 18 low-contrast spheres (6 sizes × 3 contrast levels) was scanned on a 192-slice CT scanner at five dose levels (CTDIvol = 27, 13.5, 6.8, 3.4, and 1.7 mGy), each repeated 100 times. Images were reconstructed using both filtered-backprojection (FBP) and an iterative reconstruction (IR) method (ADMIRE, Siemens). A 3D volume of interest (VOI) around each sphere was extracted and placed side-by-side with a signal-absent VOI to create a 2-alternative forced choice (2AFC) trial. Sixteen 2AFC studies were generated, each with 100 trials, to evaluate the impact of radiation dose, lesion size and contrast, and reconstruction methods on object detection. In total, 1600 trials were presented to both model and human observers. Three medical physicists acted as human observers and were allowed to page through the 3D volumes to make a decision for each 2AFC trial. The human observer performance was compared with the performance of a multislice channelized Hotelling observer (CHO_MS), which integrates multislice image data, and with the performance of previously validated CHO, which operates on static 2D images (CHO_2D). For comparison, the same 16 2AFC studies were also performed in a 2D viewing mode by the human observers and compared with the multislice viewing performance and the two CHO models.Human observer performance was well correlated with the CHO_2D performance in the 2D viewing mode [Pearson product-moment correlation coefficient R = 0.972, 95% confidence interval (CI): 0.919 to 0.990] and with the CHO_MS performance in the multislice viewing mode (R = 0.952, 95% CI: 0.865 to 0.984). The CHO_2D performance, calculated from the 2D viewing mode, also had a strong correlation with human observer performance in the multislice viewing mode (R = 0.957, 95% CI: 879 to 0.985). Human observer performance varied between the multislice and 2D modes. One reader performed better in the multislice mode (P = 0.013); whereas the other two readers showed no significant difference between the two viewing modes (P = 0.057 and P = 0.38).A 2D CHO model is highly correlated with human observer performance in detecting spherical low contrast objects in multislice viewing of CT images. This finding provides some evidence for the use of a simpler, 2D CHO to assess image quality in clinically relevant CT tasks where multislice viewing is used.

SUBMITTER: Yu L

PROVIDER: S-EPMC5553707 | biostudies-other | 2017 Aug

REPOSITORIES: biostudies-other

ACCESS DATA

Similar Datasets

Project description:Efficient optimization of CT protocols demands a quantitative approach to predicting human observer performance on specific tasks at various scan and reconstruction settings. The goal of this work was to investigate how well a channelized Hotelling observer (CHO) can predict human observer performance on 2-alternative forced choice (2AFC) lesion-detection tasks at various dose levels and two different reconstruction algorithms: a filtered-backprojection (FBP) and an iterative reconstruction (IR) method.A 35 × 26 cm(2) torso-shaped phantom filled with water was used to simulate an average-sized patient. Three rods with different diameters (small: 3 mm; medium: 5 mm; large: 9 mm) were placed in the center region of the phantom to simulate small, medium, and large lesions. The contrast relative to background was -15 HU at 120 kV. The phantom was scanned 100 times using automatic exposure control each at 60, 120, 240, 360, and 480 quality reference mAs on a 128-slice scanner. After removing the three rods, the water phantom was again scanned 100 times to provide signal-absent background images at the exact same locations. By extracting regions of interest around the three rods and on the signal-absent images, the authors generated 21 2AFC studies. Each 2AFC study had 100 trials, with each trial consisting of a signal-present image and a signal-absent image side-by-side in randomized order. In total, 2100 trials were presented to both the model and human observers. Four medical physicists acted as human observers. For the model observer, the authors used a CHO with Gabor channels, which involves six channel passbands, five orientations, and two phases, leading to a total of 60 channels. The performance predicted by the CHO was compared with that obtained by four medical physicists at each 2AFC study.The human and model observers were highly correlated at each dose level for each lesion size for both FBP and IR. The Pearson's product-moment correlation coefficients were 0.986 [95% confidence interval (CI): 0.958-0.996] for FBP and 0.985 (95% CI: 0.863-0.998) for IR. Bland-Altman plots showed excellent agreement for all dose levels and lesions sizes with a mean absolute difference of 1.0% ± 1.1% for FBP and 2.1% ± 3.3% for IR.Human observer performance on a 2AFC lesion detection task in CT with a uniform background can be accurately predicted by a CHO model observer at different radiation dose levels and for both FBP and IR methods.

Project description:A human observer study was performed for a signal detection task for the case of fan-beam x-ray computed tomography. Hotelling observer (HO) performance was calculated for the same detection task without the use of efficient channels. By considering the full image covariance produced by the filtered backprojection (FBP) algorithm and avoiding the use of channels in the computation of HO performance, the authors establish an absolute upper bound on signal detectability. Therefore, this study serves as a baseline for relating human and ideal observer performance in the case of fan-beam CT.Eight human observers participated in a two-alternative forced choice experiment where the signal of interest was a small simulated ellipsoid in the presence of independent, identically distributed Gaussian detector noise. Theoretical performance of the HO, which is equivalent to the ideal observer in this case (see Sec. 13.2.12 in Barrett and Myers [Foundations of Image Science (Wiley, Hoboken, NJ, 2004)], was also computed and compared to the performance of the human observers. In addition to a reference FBP implementation, two FBP implementations with inherent loss of HO signal detectability (e.g., by apodizing the ramp filter) were also investigated. Each of these latter two implementations takes the form of a discrete-to-discrete linear operator (i.e., a matrix), which has a nontrivial null-space resulting in the loss of detectability.Estimated observer detectability index (d(A)) values for the human observers and SNR values for the HO were obtained. While Hanning filtering in the FBP implementation with a cutoff frequency of 1/4 of the Nyquist frequency reduces HO SNR (due to the reconstruction matrix's nontrivial null-space), this filtering was shown to consistently improve human observer performance. By contrast, increasing the image pixel size was seen to have a comparable effect on both the HO and the human observers, degrading performance.These results, which characterize HO and human observer performance for a signal detection task in fan-beam FBP noise, form a basis for applying model observer metrics to fan-beam CT when knowledge of the full image-domain noise statistics is important. Further, by calculating HO performance without relying on channels, these results are particularly relevant when an information theoretic approach is considered, e.g., in optimization of the image reconstruction algorithm with respect to preservation of signal detectability. Finally, the HO (which is here equivalent to the ideal observer) provides an absolute upper bound on detection performance, and our results therefore provide insight into the performance of human observers relative to the optimum for two different cases wherein ideal observer performance is compromised through degradation of the data quality. In one case (regularization), human performance is improved to practically ideal performance, and in the other (larger pixel size), ideal and human observer performance are approximately degraded equivalently.

Project description:The purpose of this study was to investigate the correlation between model observer and human observer performance in CT imaging for the task of lesion detection and localization when the lesion location is uncertain.Two cylindrical rods (3-mm and 5-mm diameters) were placed in a 35×26 cm torso-shaped water phantom to simulate lesions with -15 HU contrast at 120 kV. The phantom was scanned 100 times on a 128-slice CT scanner at each of four dose levels (CTDIvol=5.7, 11.4, 17.1, and 22.8 mGy). Regions of interest (ROIs) around each lesion were extracted to generate images with signal-present, with each ROI containing 128×128 pixels. Corresponding ROIs of signal-absent images were generated from images without lesion mimicking rods. The location of the lesion (rod) in each ROI was randomly distributed by moving the ROIs around each lesion. Human observer studies were performed by having three trained observers identify the presence or absence of lesions, indicating the lesion location in each image and scoring confidence for the detection task on a 6-point scale. The same image data were analyzed using a channelized Hotelling model observer (CHO) with Gabor channels. Internal noise was added to the decision variables for the model observer study. Area under the curve (AUC) of ROC and localization ROC (LROC) curves were calculated using a nonparametric approach. The Spearman's rank order correlation between the average performance of the human observers and the model observer performance was calculated for the AUC of both ROC and LROC curves for both the 3- and 5-mm diameter lesions.In both ROC and LROC analyses, AUC values for the model observer agreed well with the average values across the three human observers. The Spearman's rank order correlation values for both ROC and LROC analyses for both the 3- and 5-mm diameter lesions were all 1.0, indicating perfect rank ordering agreement of the figures of merit (AUC) between the average performance of the human observers and the model observer performance.In CT imaging of different sizes of low-contrast lesions (-15 HU), the performance of CHO with Gabor channels was highly correlated with human observer performance for the detection and localization tasks with uncertain lesion location in CT imaging at four clinically relevant dose levels. This suggests the ability of Gabor CHO model observers to meaningfully assess CT image quality for the purpose of optimizing scan protocols and radiation dose levels in detection and localization tasks for low-contrast lesions.

Project description:Mathematical model observers are intended for efficient assessment of diagnostic image quality, but model-observer studies often are not representative of clinical realities. Model observers based on a visual-search (VS) paradigm may allow for greater clinical relevance. The author has compared the performances of several VS model observers with those of human observers and an existing scanning model observer for a study involving nodule detection and localization in simulated Tc-99m single-photon emission computed tomography (SPECT) lung volumes.A localization receiver operating characteristic (LROC) study compared two iterative SPECT reconstruction strategies: an all-corrections (AllC) strategy with compensations for attenuation, scatter, and distance-dependent camera resolution and an "RC" strategy with resolution compensation only. Nodules in the simulation phantom were of three different relative contrasts. Observers in the study had access to the coronal, sagittal, and transverse displays of the reconstructed volumes. Three human observers each read 50 training volumes and 100 test volumes per reconstruction strategy. The same images were analyzed by a channelized nonprewhitening (CNPW) scanning observer and two VS observers. The VS observers implemented holistic search processes that identified focal points of Tc-99m uptake for subsequent analysis by the CNPW scanning model. The level of prior knowledge about the background structure in the images was a study variable for the model observers. Performance was scored by area under the LROC curve.The average human-observer performances were respectively 0.67 ± 0.04 and 0.61 ± 0.03 for the RC and AllC strategies. Given approximate knowledge about the background structure, both VS models scored 0.69 ± 0.08 (RC) and 0.66 ± 0.08 (AllC). The scanning observer reversed the strategy ranking in scoring 0.73 ± 0.08 with the AllC strategy and 0.64 ± 0.08 with the RC strategy. The VS observers exhibited less sensitivity to variations in background knowledge compared to the scanning observer.The VS framework has the potential to increase the clinical similitude of model-observer studies and to enhance the ability of existing model observers to quantitatively predict human-observer performance.

Project description:IntroductionTo compare the diagnostic accuracy of contrast-enhanced 3D(dimensional) T1-weighted sampling perfection with application-optimized contrasts by using different flip angle evolutions (T1-SPACE), 2D fluid attenuated inversion recovery (FLAIR) images and 2D contrast-enhanced T1-weighted image in detection of leptomeningeal metastasis except for invasive procedures such as a CSF tapping.Materials and methodsThree groups of patients were included retrospectively for 9 months (from 2013-04-01 to 2013-12-31). Group 1 patients with positive malignant cells in CSF cytology (n = 22); group 2, stroke patients with steno-occlusion in ICA or MCA (n = 16); and group 3, patients with negative results on MRI, whose symptom were dizziness or headache (n = 25). A total of 63 sets of MR images are separately collected and randomly arranged: (1) CE 3D T1-SPACE; (2) 2D FLAIR; and (3) CE T1-GRE using a 3-Tesla MR system. A faculty neuroradiologist with 8-year-experience and another 2nd grade trainee in radiology reviewed each MR image- blinded by the results of CSF cytology and coded their observations as positives or negatives of leptomeningeal metastasis. The CSF cytology result was considered as a gold standard. Sensitivity and specificity of each MR images were calculated. Diagnostic accuracy was compared using a McNemar's test. A Cohen's kappa analysis was performed to assess inter-observer agreements.ResultsDiagnostic accuracy was not different between 3D T1-SPACE and CSF cytology by both raters. However, the accuracy test of 2D FLAIR and 2D contrast-enhanced T1-weighted GRE was inconsistent by the two raters. The Kappa statistic results were 0.657 (3D T1-SPACE), 0.420 (2D FLAIR), and 0.160 (2D contrast-enhanced T1-weighted GRE). The 3D T1-SPACE images showed the highest inter-observer agreements between the raters.ConclusionsCompared to 2D FLAIR and 2D contrast-enhanced T1-weighted GRE, contrast-enhanced 3D T1 SPACE showed a better detection rate of leptomeningeal metastasis.

Dataset Information

Correlation between a 2D channelized Hotelling observer and human observers in a low-contrast detection task with multislice reading in CT.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets