Dataset Information

Training the ACRIN 6666 Investigators and effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS ultrasound feature analysis.

ABSTRACT: Qualification tasks in mammography and breast ultrasound were developed for the American College of Radiology Imaging Network (ACRIN) 6666 Investigators. We sought to assess the effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS feature analysis among a subset of these experienced observers.After a 1-hour didactic session on BI-RADS: Ultrasound, an interpretive skills quiz set of 70 orthogonal sets of breast ultrasound images including 25 (36%) malignancies was presented to 100 experienced breast imaging observers. Thirty-five observers reviewed the quiz set twice: first without and then with immediate feedback of consensus feature analysis, management recommendations, and pathologic truth. Observer performance (sensitivity, specificity, area under the curve [AUC]) was calculated without feedback and with feedback. Kappas were determined for agreement on feature analysis and assessments.For 35 observers without feedback, the mean sensitivity was 89% (range, 68-100%); specificity, 62% (range, 42-82%); and AUC, 82% (range, 73-89%). With feedback, the mean sensitivity was 93% (range, 80-100%; mean increase, 4%; range of increase, 0-12%; p < 0.0001), the mean specificity was 61% (range, 45-73%; mean decrease, 1%; range of change, -18% to 11%; p = 0.19), and the mean AUC was 84% (range, 78-90%; mean increase, 2%; range of change, -3% to 9%; p < 0.0001). Three breast imagers in the lowest quartile of initial performance showed the greatest improvement in sensitivity with no change or improvement in AUC. The kappa values for feature analysis did not change, but there was improved agreement about final assessments, with the kappa value increasing from 0.53 (SE, 0.02) without feedback to 0.59 (SE, 0.02) with feedback (p < 0.0001).Most experienced breast imagers showed excellent breast ultrasound interpretive skills. Immediate feedback of consensus BI-RADS: Ultrasound features and histopathologic results improved performance in ultrasound interpretation across all experience variables.

SUBMITTER: Berg WA

PROVIDER: S-EPMC3891891 | biostudies-literature | 2012 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Training the ACRIN 6666 Investigators and effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS ultrasound feature analysis.

Berg Wendie A WA Blume Jeffrey D JD Cormack Jean B JB Mendelson Ellen B EB

AJR. American journal of roentgenology 20120701 1

<h4>Objective</h4>Qualification tasks in mammography and breast ultrasound were developed for the American College of Radiology Imaging Network (ACRIN) 6666 Investigators. We sought to assess the effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS feature analysis among a subset of these experienced observers.<h4>Materials and methods</h4>After a 1-hour didactic session on BI-RADS: Ultrasound, an interpretive skills quiz set of 70 orthogonal sets of breast ...[more]

PMID: 22733916

Similar Datasets

Project description:Background The classification of Breast Imaging Reporting and Data System 4A (BI-RADS 4A) lesions is mostly based on the personal experience of doctors and lacks specific and clear classification standards. The development of artificial intelligence (AI) provides a new method for BI-RADS categorisation. We analysed the ultrasonic morphological and texture characteristics of BI-RADS 4A benign and malignant lesions using AI, and these ultrasonic characteristics of BI-RADS 4A benign and malignant lesions were compared to examine the value of AI in the differential diagnosis of BI-RADS 4A benign and malignant lesions. Methods A total of 206 lesions of BI-RADS 4A examined using ultrasonography were analysed retrospectively, including 174 benign lesions and 32 malignant lesions. All of the lesions were contoured manually, and the ultrasonic morphological and texture features of the lesions, such as circularity, height-to-width ratio, margin spicules, margin coarseness, margin indistinctness, margin lobulation, energy, entropy, grey mean, internal calcification and angle between the long axis of the lesion and skin, were calculated using grey level gradient co-occurrence matrix analysis. Differences between benign and malignant lesions of BI-RADS 4A were analysed. Results Significant differences in margin lobulation, entropy, internal calcification and ALS were noted between the benign group and malignant group (P?=?0.013, 0.045, 0.045, and 0.002, respectively). The malignant group had more margin lobulations and lower entropy compared with the benign group, and the benign group had more internal calcifications and a greater angle between the long axis of the lesion and skin compared with the malignant group. No significant differences in circularity, height-to-width ratio, margin spicules, margin coarseness, margin indistinctness, energy, and grey mean were noted between benign and malignant lesions. Conclusions Compared with the naked eye, AI can reveal more subtle differences between benign and malignant BI-RADS 4A lesions. These results remind us carefully observation of the margin and the internal echo is of great significance. With the help of morphological and texture information provided by AI, doctors can make a more accurate judgment on such atypical benign and malignant lesions.

Project description:To evaluate the positive predictive values (PPVs) of Breast Imaging and Reporting Data Systems (BI-RADS) assessment categories for breast magnetic resonance (MR) imaging and to identify the BI-RADS MR imaging lesion features most predictive of cancer.This institutional review board-approved HIPAA-compliant prospective multicenter study was performed with written informed consent. Breast MR imaging studies of the contralateral breast in women with a recent diagnosis of breast cancer were prospectively evaluated. Contralateral breast MR imaging BI-RADS assessment categories, morphologic descriptors for foci, masses, non-masslike enhancement (NMLE), and kinetic features were assessed for predictive values for malignancy. PPV of each imaging characteristic of interest was estimated, and logistic regression analysis was used to examine the predictive ability of combinations of characteristics.Of 969 participants, 71.3% had a BI-RADS category 1 or 2 assessment; 10.9%, a BI-RADS category 3 assessment; 10.0%, a BI-RADS category 4 or 5 assessment; and 7.7%, a BI-RADS category 0 assessment on the basis of initial MR images. Thirty-one cancers were detected with MR imaging. Overall PPV for BI-RADS category 4 and 5 lesions was 0.278, with 17 cancers in patients with a BI-RADS category 4 lesion (PPV, 0.205) and 10 cancers in patients with a BI-RADS category 5 lesion (PPV, 0.714). Of the cancers, one was a focus, 17 were masses, and 13 were NMLEs. For masses, irregular shape, irregular margins, spiculated margins, and marked internal enhancement were most predictive of malignancy. For NMLEs, ductal, clumped, and reticular or dendritic enhancement were the features most frequently seen with malignancy. Kinetic enhancement features were less predictive of malignancy than were morphologic features.Standardized terminology of the BI-RADS lexicon enables quantification of the likelihood of malignancy for MR imaging-detected lesions through careful evaluation of lesion features. In particular, BI-RADS assessment categories and morphologic descriptors for masses and NMLE were useful in estimating the probability of cancer.

Project description:We developed a machine learning model based on radiomics to predict the BI-RADS category of ultrasound-detected suspicious breast lesions and support medical decision-making towards short-interval follow-up versus tissue sampling. From a retrospective 2015-2019 series of ultrasound-guided core needle biopsies performed by four board-certified breast radiologists using six ultrasound systems from three vendors, we collected 821 images of 834 suspicious breast masses from 819 patients, 404 malignant and 430 benign according to histopathology. A balanced image set of biopsy-proven benign (n = 299) and malignant (n = 299) lesions was used for training and cross-validation of ensembles of machine learning algorithms supervised during learning by histopathological diagnosis as a reference standard. Based on a majority vote (over 80% of the votes to have a valid prediction of benign lesion), an ensemble of support vector machines showed an ability to reduce the biopsy rate of benign lesions by 15% to 18%, always keeping a sensitivity over 94%, when externally tested on 236 images from two image sets: (1) 123 lesions (51 malignant and 72 benign) obtained from two ultrasound systems used for training and from a different one, resulting in a positive predictive value (PPV) of 45.9% (95% confidence interval 36.3-55.7%) versus a radiologists' PPV of 41.5% (p < 0.005), combined with a 98.0% sensitivity (89.6-99.9%); (2) 113 lesions (54 malignant and 59 benign) obtained from two ultrasound systems from vendors different from those used for training, resulting into a 50.5% PPV (40.4-60.6%) versus a radiologists' PPV of 47.8% (p < 0.005), combined with a 94.4% sensitivity (84.6-98.8%). Errors in BI-RADS 3 category (i.e., assigned by the model as BI-RADS 4) were 0.8% and 2.7% in the Testing set I and II, respectively. The board-certified breast radiologist accepted the BI-RADS classes assigned by the model in 114 masses (92.7%) and modified the BI-RADS classes of 9 breast masses (7.3%). In six of nine cases, the model performed better than the radiologist did, since it assigned a BI-RADS 3 classification to histopathology-confirmed benign masses that were classified as BI-RADS 4 by the radiologist.

Project description:Background Classification of lung cancer screening CT scans depends on measurement of lung nodule size. Information about interobserver agreement is limited. Purpose To assess interobserver agreement in the measurements and American College of Radiology Lung CT Screening Reporting and Data System (Lung-RADS) classifications of solid lung nodules detected at lung cancer screening using manual measurements of average diameter and computer-aided semiautomated measurements of average diameter and volume (CT volumetry). Materials and Methods Two radiologists and one radiology resident retrospectively measured lung nodules from screening CT scans obtained between September 2016 and June 2018 with a Lung-RADS (version 1.0) classification of 2, 3, 4A, or 4B in the clinical setting. Average manual diameter and semiautomated computer-aided diameter and volume measurements were converted to the corresponding Lung-RADS categories. Interobserver agreement in raw measurements was assessed using intraclass correlation and Bland-Altman indexes, and interobserver agreement in Lung-RADS classification was assessed using bi-rater κ. Results One hundred twenty patients (mean age, 63 years ± 6 [standard deviation]; 67 women) were evaluated. All manual, semiautomated diameter, and semiautomated volume measurements were obtained by all three readers in 120 of 147 nodules (82%). Intraclass correlation coefficients were greater than or equal to 0.95 for all reader pairs using all measurement methods and were highest using volumetry. Bias and 95% limits of agreement for average diameter were smaller with semiautomated measurements than with manual measurements. κ values across all Lung-RADS classifications were greater than or equal to 0.81, with the lowest being for manual measurements and the highest being for volumetric measurements. Forty-three of 120 (36%) of the nodules were classified into a lower Lung-RADS category on the basis of volumetry compared with using manual diameter measurements by at least one reader, whereas the reverse occurred for four of 120 (3%) of the nodules. Conclusion Interobserver agreement was high with manual diameter measurements and increased with semiautomated CT volumetric measurements. Semiautomated CT volumetry enabled classification of more nodules into lower Lung CT Screening Reporting and Data System categories than manual or semiautomated diameter measurements. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Nishino in this issue.

Dataset Information

Training the ACRIN 6666 Investigators and effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS ultrasound feature analysis.

Publications

Training the ACRIN 6666 Investigators and effects of feedback on breast ultrasound interpretive performance and agreement in BI-RADS ultrasound feature analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets