Dataset Information

National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium.

ABSTRACT: Purpose To establish contemporary performance benchmarks for diagnostic digital mammography with use of recent data from the Breast Cancer Surveillance Consortium (BCSC). Materials and Methods Institutional review board approval was obtained for active or passive consenting processes or to obtain a waiver of consent to enroll participants, link data, and perform analyses. Data were obtained from six BCSC registries (418 radiologists, 92 radiology facilities). Mammogram indication and assessments were prospectively collected for women undergoing diagnostic digital mammography and linked with cancer diagnoses from state cancer registries. The study included 401 548 examinations conducted from 2007 to 2013 in 265 360 women. Results Overall diagnostic performance measures were as follows: cancer detection rate, 34.7 per 1000 (95% confidence interval [CI]: 34.1, 35.2); abnormal interpretation rate, 12.6% (95% CI: 12.5%, 12.7%); positive predictive value (PPV) of a biopsy recommendation (PPV2), 27.5% (95% CI: 27.1%, 27.9%); PPV of biopsies performed (PPV3), 30.4% (95% CI: 29.9%, 30.9%); false-negative rate, 4.8 per 1000 (95% CI: 4.6, 5.0); sensitivity, 87.8% (95% CI: 87.3%, 88.4%); and specificity, 90.5% (95% CI: 90.4%, 90.6%). Among cancers detected, 63.4% were stage 0 or 1 cancers, 45.6% were minimal cancers, the mean size of invasive cancers was 21.2 mm, and 69.6% of invasive cancers were node negative. Performance metrics varied widely across diagnostic indications, with cancer detection rate (64.5 per 1000) and abnormal interpretation rate (18.7%) highest for diagnostic mammograms obtained to evaluate a breast problem with a lump. Compared with performance during the screen-film mammography era, diagnostic digital performance showed increased abnormal interpretation and cancer detection rates and decreasing PPVs, with less than 70% of radiologists within acceptable ranges for PPV2 and PPV3. Conclusion These performance measures can serve as national benchmarks that may help transform the marked variation in radiologists' diagnostic performance into targeted quality improvement efforts. © RSNA, 2017 Online supplemental material is available for this article.

SUBMITTER: Sprague BL

PROVIDER: S-EPMC5375630 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium.

Sprague Brian L BL Arao Robert F RF Miglioretti Diana L DL Henderson Louise M LM Buist Diana S M DS Onega Tracy T Rauscher Garth H GH Lee Janie M JM Tosteson Anna N A AN Kerlikowske Karla K Lehman Constance D CD

Radiology 20170228 1

Purpose To establish contemporary performance benchmarks for diagnostic digital mammography with use of recent data from the Breast Cancer Surveillance Consortium (BCSC). Materials and Methods Institutional review board approval was obtained for active or passive consenting processes or to obtain a waiver of consent to enroll participants, link data, and perform analyses. Data were obtained from six BCSC registries (418 radiologists, 92 radiology facilities). Mammogram indication and assessments ...[more]

PMID: 28244803

Similar Datasets

Project description:Purpose To establish performance benchmarks for modern screening digital mammography and assess performance trends over time in U.S. community practice. Materials and Methods This HIPAA-compliant, institutional review board-approved study measured the performance of digital screening mammography interpreted by 359 radiologists across 95 facilities in six Breast Cancer Surveillance Consortium (BCSC) registries. The study included 1 682 504 digital screening mammograms performed between 2007 and 2013 in 792 808 women. Performance measures were calculated according to the American College of Radiology Breast Imaging Reporting and Data System, 5th edition, and were compared with published benchmarks by the BCSC, the National Mammography Database, and performance recommendations by expert opinion. Benchmarks were derived from the distribution of performance metrics across radiologists and were presented as 50th (median), 10th, 25th, 75th, and 90th percentiles, with graphic presentations using smoothed curves. Results Mean screening performance measures were as follows: abnormal interpretation rate (AIR), 11.6 (95% confidence interval [CI]: 11.5, 11.6); cancers detected per 1000 screens, or cancer detection rate (CDR), 5.1 (95% CI: 5.0, 5.2); sensitivity, 86.9% (95% CI: 86.3%, 87.6%); specificity, 88.9% (95% CI: 88.8%, 88.9%); false-negative rate per 1000 screens, 0.8 (95% CI: 0.7, 0.8); positive predictive value (PPV) 1, 4.4% (95% CI: 4.3%, 4.5%); PPV2, 25.6% (95% CI: 25.1%, 26.1%); PPV3, 28.6% (95% CI: 28.0%, 29.3%); cancers stage 0 or 1, 76.9%; minimal cancers, 57.7%; and node-negative invasive cancers, 79.4%. Recommended CDRs were achieved by 92.1% of radiologists in community practice, and 97.1% achieved recommended ranges for sensitivity. Only 59.0% of radiologists achieved recommended AIRs, and only 63.0% achieved recommended levels of specificity. Conclusion The majority of radiologists in the BCSC surpass cancer detection recommendations for screening mammography; however, AIRs continue to be higher than the recommended rate for almost half of radiologists interpreting screening mammograms. © RSNA, 2016 Online supplemental material is available for this article.

Project description:To develop criteria to identify thresholds for the minimally acceptable performance of physicians interpreting diagnostic mammography studies.In an institutional review board-approved HIPAA-compliant study, an Angoff approach was used to set criteria for identifying minimally acceptable interpretive performance for both workup after abnormal screening examinations and workup of a breast lump. Normative data from the Breast Cancer Surveillance Consortium (BCSC) was used to help the expert radiologist identify the impact of cut points. Simulations, also using data from the BCSC, were used to estimate the expected clinical impact from the recommended performance thresholds.Final cut points for workup of abnormal screening examinations were as follows: sensitivity, less than 80%; specificity, less than 80% or greater than 95%; abnormal interpretation rate, less than 8% or greater than 25%; positive predictive value (PPV) of biopsy recommendation (PPV2), less than 15% or greater than 40%; PPV of biopsy performed (PPV3), less than 20% or greater than 45%; and cancer diagnosis rate, less than 20 per 1000 interpretations. Final cut points for workup of a breast lump were as follows: sensitivity, less than 85%; specificity, less than 83% or greater than 95%; abnormal interpretation rate, less than 10% or greater than 25%; PPV2, less than 25% or greater than 50%; PPV3, less than 30% or greater than 55%; and cancer diagnosis rate, less than 40 per 1000 interpretations. If underperforming physicians moved into the acceptable range after remedial training, the expected result would be (a) diagnosis of an additional 86 cancers per 100,000 women undergoing workup after screening examinations, with a reduction in the number of false-positive examinations by 1067 per 100,000 women undergoing this workup, and (b) diagnosis of an additional 335 cancers per 100,000 women undergoing workup of a breast lump, with a reduction in the number of false-positive examinations by 634 per 100,000 women undergoing this workup.Interpreting physicians who fall outside one or more of the identified cut points should be reviewed in the context of an overall assessment of all their performance measures and their specific practice setting to determine if remedial training is indicated.

Project description:BackgroundOur objective was to perform a systematic review and meta-analysis comparing the breast cancer detection rate (CDR), invasive CDR, recall rate, and positive predictive value 1 (PPV1) of digital mammography (DM) alone, combined digital breast tomosynthesis (DBT) and DM, combined DBT and synthetic 2-dimensional mammography (S2D), and DBT alone.MethodsMEDLINE and Embase were searched until April 2020 to identify comparative design studies reporting on patients undergoing routine breast cancer screening. Random effects model proportional meta-analyses estimated CDR, invasive CDR, recall rate, and PPV1. Meta-regression modeling was used to compare imaging modalities. All statistical tests were 2-sided.ResultsForty-two studies reporting on 2 606 296 patients (13 003 breast cancer cases) were included. CDR was highest in combined DBT and DM (6.36 per 1000 screened, 95% confidence interval [CI] = 5.62 to 7.14, P < .001), and combined DBT and S2D (7.40 per 1000 screened, 95% CI = 6.49 to 8.37, P < .001) compared with DM alone (4.68 per 1000 screened, 95% CI = 4.28 to 5.11). Invasive CDR was highest in combined DBT and DM (4.53 per 1000 screened, 95% CI = 3.97 to 5.12, P = .003) and combined DBT and S2D (5.68 per 1000 screened, 95% CI = 4.43 to 7.09, P < .001) compared with DM alone (3.42 per 1000 screened, 95% CI = 3.02 to 3.83). Recall rate was lowest in combined DBT and S2D (42.3 per 1000 screened, 95% CI = 37.4 to 60.4, P<.001). PPV1 was highest in combined DBT and DM (10.0%, 95% CI = 8.0% to 12.0%, P = .004), and combined DBT and S2D (16.0%, 95% CI = 10.0% to 23.0%, P < .001), whereas no difference was detected for DBT alone (7.0%, 95% CI = 6.0% to 8.0%, P = .75) compared with DM alone (7.0%, 95.0% CI = 5.0% to 8.0%).ConclusionsOur findings provide evidence on key performance metrics for DM, DBT alone, combined DBT and DM, and combined DBT and S2D, which may inform optimal application of these modalities for breast cancer screening.

Project description:PurposeTo compare radiologists' diagnostic accuracy and recall rates for breast tomosynthesis combined with digital mammography versus digital mammography alone.Materials and methodsInstitutional review board approval was obtained at each accruing institution. Participating women gave written informed consent. Mediolateral oblique and craniocaudal digital mammographic and tomosynthesis images of both breasts were obtained from 1192 subjects. Two enriched reader studies were performed to compare digital mammography with tomosynthesis against digital mammography alone. Study 1 comprised 312 cases (48 cancer cases) with images read by 12 radiologists; study 2, 312 cases (51 cancer cases) with 15 radiologists. Study 1 readers recorded only that an abnormality requiring recall was present; study 2 readers had additional training and recorded both lesion type and location. Diagnostic accuracy was compared with receiver operating characteristic analysis. Recall rates of noncancer cases, sensitivity, specificity, and positive and negative predictive values determined by analyzing Breast Imaging Reporting and Data System scores were compared for the two methods.ResultsDiagnostic accuracy for combined tomosynthesis and digital mammography was superior to that of digital mammography alone. Average difference in area under the curve in study 1 was 7.2% (95% confidence interval [CI]: 3.7%, 10.8%; P < .001) and in study 2 was 6.8% (95% CI: 4.1%, 9.5%; P < .001). All 27 radiologists increased diagnostic accuracy with addition of tomosynthesis. Recall rates for noncancer cases for all readers significantly decreased with addition of tomosynthesis (range, 6%-67%; P < .001 for 25 readers, P < .03 for all readers). Increased sensitivity was largest for invasive cancers: 15% and 22% in studies 1 and 2 versus 3% for in situ cancers in both studies.ConclusionAddition of tomosynthesis to digital mammography offers the dual benefit of significantly increased diagnostic accuracy and significantly reduced recall rates for noncancer cases.Supplemental materialhttp://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.12120674/-/DC1.

Project description:After the US Food and Drug Administration (FDA) approved computer-aided detection (CAD) for mammography in 1998, and the Centers for Medicare and Medicaid Services (CMS) provided increased payment in 2002, CAD technology disseminated rapidly. Despite sparse evidence that CAD improves accuracy of mammographic interpretations and costs over $400 million a year, CAD is currently used for most screening mammograms in the United States.To measure performance of digital screening mammography with and without CAD in US community practice.We compared the accuracy of digital screening mammography interpreted with (n = 495 818) vs without (n = 129 807) CAD from 2003 through 2009 in 323 973 women. Mammograms were interpreted by 271 radiologists from 66 facilities in the Breast Cancer Surveillance Consortium. Linkage with tumor registries identified 3159 breast cancers in 323 973 women within 1 year of the screening.Mammography performance (sensitivity, specificity, and screen-detected and interval cancers per 1000 women) was modeled using logistic regression with radiologist-specific random effects to account for correlation among examinations interpreted by the same radiologist, adjusting for patient age, race/ethnicity, time since prior mammogram, examination year, and registry. Conditional logistic regression was used to compare performance among 107 radiologists who interpreted mammograms both with and without CAD.Screening performance was not improved with CAD on any metric assessed. Mammography sensitivity was 85.3% (95% CI, 83.6%-86.9%) with and 87.3% (95% CI, 84.5%-89.7%) without CAD. Specificity was 91.6% (95% CI, 91.0%-92.2%) with and 91.4% (95% CI, 90.6%-92.0%) without CAD. There was no difference in cancer detection rate (4.1 in 1000 women screened with and without CAD). Computer-aided detection did not improve intraradiologist performance. Sensitivity was significantly decreased for mammograms interpreted with vs without CAD in the subset of radiologists who interpreted both with and without CAD (odds ratio, 0.53; 95% CI, 0.29-0.97).Computer-aided detection does not improve diagnostic accuracy of mammography. These results suggest that insurers pay more for CAD with no established benefit to women.

Project description:ImportanceMany US radiologists have screening mammography recall rates above the expert-recommended threshold of 12%. The influence of digital breast tomosynthesis (DBT) on the distribution of radiologist recall rates is uncertain.ObjectiveTo evaluate radiologists' recall and cancer detection rates before and after beginning interpretation of DBT examinations.Design, setting, and participantsThis cohort study included 198 radiologists from 104 radiology facilities in the Breast Cancer Surveillance Consortium who interpreted 251?384 DBT and 2?000?681 digital mammography (DM) screening examinations from 2009 to 2017, including 126 radiologists (63.6%) who interpreted DBT examinations during the study period and 72 (36.4%) who exclusively interpreted DM examinations (to adjust for secular trends). Data were analyzed from April 2018 to July 2019.ExposuresDigital breast tomosynthesis and DM screening examinations.Main outcomes and measuresRecall rate and cancer detection rate.ResultsA total of 198 radiologists interpreted 2?252?065 DM and DBT examinations (2?000?681 [88.8%] DM examinations; 251?384 [11.2%] DBT examinations; 710?934 patients [31.6%] aged 50-59 years; 1?448?981 [64.3%] non-Hispanic white). Among the 126 radiologists (63.6%) who interpreted DBT examinations, 83 (65.9%) had unadjusted DM recall rates of no more than 12% before using DBT, with a median (interquartile range) recall rate of 10.0% (7.5%-13.0%). On DBT examinations, 96 (76.2%) had an unadjusted recall rate of no more than 12%, with a median (interquartile range) recall rate of 8.8% (6.3%-11.3%). A secular trend in recall rate was observed, with the multivariable-adjusted risk of recall on screening examinations declining by 1.2% (95% CI, 0.9%-1.5%) per year. After adjusting for examination characteristics and secular trends, recall rates were 15% lower on DBT examinations compared with DM examinations interpreted before DBT use (relative risk, 0.85; 95% CI, 0.83-0.87). Adjusted recall rates were significantly lower on DBT examinations compared with DM examinations interpreted before DBT use for 45 radiologists (35.7%) and significantly higher for 18 (14.3%); 63 (50.0%) had no statistically significant change. The unadjusted cancer detection rate on DBT was 5.3 per 1000 examinations (95% CI, 5.0-5.7 per 1000 examinations) compared with 4.7 per 1000 examinations (95% CI, 4.6-4.8 per 1000 examinations) on DM examinations interpreted before DM use (multivariable-adjusted risk ratio, 1.21; 95% CI, 1.11-1.33).Conclusions and relevanceIn this study, DBT was associated with an overall decrease in recall rate and an increase in cancer detection rate. However, our results indicated that there is wide variability among radiologists, including a subset of radiologists who experienced increased recall rates on DBT examinations. Radiology practices should audit radiologist DBT screening performance and consider additional DBT training for radiologists whose performance does not improve as expected.

Dataset Information

National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium.

Publications

National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets