Dataset Information

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists.

ABSTRACT: BACKGROUND:Artificial intelligence (AI) systems performing at radiologist-like levels in the evaluation of digital mammography (DM) would improve breast cancer screening accuracy and efficiency. We aimed to compare the stand-alone performance of an AI system to that of radiologists in detecting breast cancer in DM. METHODS:Nine multi-reader, multi-case study datasets previously used for different research purposes in seven countries were collected. Each dataset consisted of DM exams acquired with systems from four different vendors, multiple radiologists' assessments per exam, and ground truth verified by histopathological analysis or follow-up, yielding a total of 2652 exams (653 malignant) and interpretations by 101 radiologists (28?296 independent interpretations). An AI system analyzed these exams yielding a level of suspicion of cancer present between 1 and 10. The detection performance between the radiologists and the AI system was compared using a noninferiority null hypothesis at a margin of 0.05. RESULTS:The performance of the AI system was statistically noninferior to that of the average of the 101 radiologists. The AI system had a 0.840 (95% confidence interval [CI] = 0.820 to 0.860) area under the ROC curve and the average of the radiologists was 0.814 (95% CI = 0.787 to 0.841) (difference 95% CI = -0.003 to 0.055). The AI system had an AUC higher than 61.4% of the radiologists. CONCLUSIONS:The evaluated AI system achieved a cancer detection accuracy comparable to an average breast radiologist in this retrospective setting. Although promising, the performance and impact of such a system in a screening setting needs further investigation.

SUBMITTER: Rodriguez-Ruiz A

PROVIDER: S-EPMC6748773 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists.

Rodriguez-Ruiz Alejandro A Lång Kristina K Gubern-Merida Albert A Broeders Mireille M Gennaro Gisella G Clauser Paola P Helbich Thomas H TH Chevalier Margarita M Tan Tao T Mertelmeier Thomas T Wallis Matthew G MG Andersson Ingvar I Zackrisson Sophia S Mann Ritse M RM Sechopoulos Ioannis I

Journal of the National Cancer Institute 20190901 9

<h4>Background</h4>Artificial intelligence (AI) systems performing at radiologist-like levels in the evaluation of digital mammography (DM) would improve breast cancer screening accuracy and efficiency. We aimed to compare the stand-alone performance of an AI system to that of radiologists in detecting breast cancer in DM.<h4>Methods</h4>Nine multi-reader, multi-case study datasets previously used for different research purposes in seven countries were collected. Each dataset consisted of DM exa ...[more]

PMID: 30834436

Similar Datasets

Project description:ObjectiveTo evaluate whether artificial intelligence (AI) for detecting breast cancer on mammography can improve the performance and time efficiency of radiologists reading mammograms.Materials and methodsA commercial deep learning-based software for mammography was validated using external data collected from 200 patients, 100 each with and without breast cancer (40 with benign lesions and 60 without lesions) from one hospital. Ten readers, including five breast specialist radiologists (BSRs) and five general radiologists (GRs), assessed all mammography images using a seven-point scale to rate the likelihood of malignancy in two sessions, with and without the aid of the AI-based software, and the reading time was automatically recorded using a web-based reporting system. Two reading sessions were conducted with a two-month washout period in between. Differences in the area under the receiver operating characteristic curve (AUROC), sensitivity, specificity, and reading time between reading with and without AI were analyzed, accounting for data clustering by readers when indicated.ResultsThe AUROC of the AI alone, BSR (average across five readers), and GR (average across five readers) groups was 0.915 (95% confidence interval, 0.876-0.954), 0.813 (0.756-0.870), and 0.684 (0.616-0.752), respectively. With AI assistance, the AUROC significantly increased to 0.884 (0.840-0.928) and 0.833 (0.779-0.887) in the BSR and GR groups, respectively (p = 0.007 and p < 0.001, respectively). Sensitivity was improved by AI assistance in both groups (74.6% vs. 88.6% in BSR, p < 0.001; 52.1% vs. 79.4% in GR, p < 0.001), but the specificity did not differ significantly (66.6% vs. 66.4% in BSR, p = 0.238; 70.8% vs. 70.0% in GR, p = 0.689). The average reading time pooled across readers was significantly decreased by AI assistance for BSRs (82.73 vs. 73.04 seconds, p < 0.001) but increased in GRs (35.44 vs. 42.52 seconds, p < 0.001).ConclusionAI-based software improved the performance of radiologists regardless of their experience and affected the reading time.

Project description:PurposeTo investigate how an artificial intelligence (AI) system performs at digital mammography (DM) from a screening population with ground truth defined by digital breast tomosynthesis (DBT), and whether AI could detect breast cancers at DM that had originally only been detected at DBT.Materials and methodsIn this secondary analysis of data from a prospective study, DM examinations from 14 768 women (mean age, 57 years), examined with both DM and DBT with independent double reading in the Malmӧ Breast Tomosynthesis Screening Trial (MBTST) (ClinicalTrials.gov: NCT01091545; data collection, 2010-2015), were analyzed with an AI system. Of 136 screening-detected cancers, 95 cancers were detected at DM and 41 cancers were detected only at DBT. The system identifies suspicious areas in the image, scored 1-100, and provides a risk score of 1 to 10 for the whole examination. A cancer was defined as AI detected if the cancer lesion was correctly localized and scored at least 62 (threshold determined by the AI system developers), therefore resulting in the highest examination risk score of 10. Data were analyzed with descriptive statistics, and detection performance was analyzed with receiver operating characteristics.ResultsThe highest examination risk score was assigned to 10% (1493 of 14 786) of the examinations. With 90.8% specificity, the AI system detected 75% (71 of 95) of the DM-detected cancers and 44% (18 of 41) of cancers at DM that had originally been detected only at DBT. The majority were invasive cancers (17 of 18).ConclusionAlmost half of the additional DBT-only screening-detected cancers in the MBTST were detected at DM with AI. AI did not reach double reading performance; however, if combined with double reading, AI has the potential to achieve a substantial portion of the benefit of DBT screening.Keywords: Computer-aided Diagnosis, Mammography, Breast, Diagnosis, Classification, Application DomainClinical trial registration no. NCT01091545© RSNA, 2021.

Project description:To determine whether cmAssist™, an artificial intelligence-based computer-aided detection (AI-CAD) algorithm, can be used to improve radiologists' sensitivity in breast cancer screening and detection. A blinded retrospective study was performed with a panel of seven radiologists using a cancer-enriched data set from 122 patients that included 90 false-negative mammograms obtained up to 5.8 years prior to diagnosis and 32 BIRADS 1 and 2 patients with a 2-year follow-up of negative diagnosis. The mammograms were performed between February 7, 2008 (earliest) and January 8, 2016 (latest), and were all originally interpreted as negative in conjunction with R2 ImageChecker CAD, version 10.0. In this study, the readers analyzed the 122 studies before and after review of cmAssist™, an AI-CAD software for mammography. The statistical significance of our findings was evaluated using Student's t test and bootstrap statistical analysis. There was a substantial and significant improvement in radiologist accuracy with use of cmAssist, as demonstrated in the 7.2% increase in the area-under-the-curve (AUC) of the receiver operating characteristic (ROC) curve with two-sided p value?<?0.01 for the reader group. All radiologists showed a significant improvement in their cancer detection rate (CDR) with the use of cmAssist (two-sided p value?=?0.030, confidence interval?=?95%). The readers detected between 25 and 71% (mean 51%) of the early cancers without assistance. With cmAssist, the overall reader CDR was 41 to 76% (mean 62%). The percentage increase in CDR for the reader panel was significant, ranging from 6 to 64% (mean 27%) with the use of cmAssist. There was less than 1% increase in the readers' false-positive recalls with use of cmAssist. With the use of cmAssist TM, there was a substantial and statistically significant improvement in radiologists' accuracy and sensitivity for detection of cancers that were originally missed. The percentage increase in CDR for the radiologists in the reader panel ranged from 6 to 64% (mean 27%) with the use of cmAssist, with negligible increase in false-positive recalls.

Project description:ObjectivesDigital breast tomosynthesis (DBT) can detect more cancers than the current standard breast screening method, digital mammography (DM); however, it can substantially increase the reading workload and thus hinder implementation in screening. Artificial intelligence (AI) might be a solution. The aim of this study was to retrospectively test different ways of using AI in a screening workflow.MethodsAn AI system was used to analyse 14,772 double-read single-view DBT examinations from a screening trial with paired DM double reading. Three scenarios were studied: if AI can identify normal cases that can be excluded from human reading; if AI can replace the second reader; if AI can replace both readers. The number of detected cancers and false positives was compared with DM or DBT double reading.ResultsBy excluding normal cases and only reading 50.5% (7460/14,772) of all examinations, 95% (121/127) of the DBT double reading detected cancers could be detected. Compared to DM screening, 27% (26/95) more cancers could be detected (p < 0.001) while keeping recall rates at the same level. With AI replacing the second reader, 95% (120/127) of the DBT double reading detected cancers could be detected-26% (25/95) more than DM screening (p < 0.001)-while increasing recall rates by 53%. AI alone with DBT has a sensitivity similar to DM double reading (p = 0.689).ConclusionAI can open up possibilities for implementing DBT screening and detecting more cancers with the total reading workload unchanged. Considering the potential legal and psychological implications, replacing the second reader with AI would probably be most the feasible approach.Key points• Breast cancer screening with digital breast tomosynthesis and artificial intelligence can detect more cancers than mammography screening without increasing screen-reading workload. • Artificial intelligence can either exclude low-risk cases from double reading or replace the second reader. • Retrospective study based on paired mammography and digital breast tomosynthesis screening data.

Dataset Information

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists.

Publications

Stand-Alone Artificial Intelligence for Breast Cancer Detection in Mammography: Comparison With 101 Radiologists.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets