Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Leibig C
PROVIDER: S-EPMC9839981 | biostudies-literature | 2022 Jul
REPOSITORIES: biostudies-literature
Publications
Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis.
Leibig Christian C Brehmer Moritz M Bunk Stefan S Byng Danalyn D Pinker Katja K Umutlu Lale L
The Lancet. Digital health 20220701 7
<h4>Background</h4>We propose a decision-referral approach for integrating artificial intelligence (AI) into the breast-cancer screening pathway, whereby the algorithm makes predictions on the basis of its quantification of uncertainty. Algorithmic assessments with high certainty are done automatically, whereas assessments with lower certainty are referred to the radiologist. This two-part AI system can triage normal mammography exams and provide post-hoc cancer detection to maintain a high degr ...[more]
Similar Datasets
Project description:BackgroundArtificial intelligence (AI) systems are proposed as a replacement of the first reader in double reading within mammography screening. We aimed to assess cancer detection accuracy of an AI system in a Danish screening population.MethodsWe retrieved a consecutive screening cohort from the Region of Southern Denmark including all participating women between Aug 4, 2014, and August 15, 2018. Screening mammograms were processed by a commercial AI system and detection accuracy was evaluated in two scenarios, Standalone AI and AI-integrated screening replacing first reader, with first reader and double reading with arbitration (combined reading) as comparators, respectively. Two AI-score cut-off points were applied by matching at mean first reader sensitivity (AIsens) and specificity (AIspec). Reference standard was histopathology-proven breast cancer or cancer-free follow-up within 24 months. Coprimary endpoints were sensitivity and specificity, and secondary endpoints were positive predictive value (PPV), negative predictive value (NPV), recall rate, and arbitration rate. Accuracy estimates were calculated using McNemar's test or exact binomial test.ResultsOut of 272,008 screening mammograms from 158,732 women, 257,671 (94.7%) with adequate image data were included in the final analyses. Sensitivity and specificity were 63.7% (95% CI 61.6%-65.8%) and 97.8% (97.7-97.8%) for first reader, and 73.9% (72.0-75.8%) and 97.9% (97.9-98.0%) for combined reading, respectively. Standalone AIsens showed a lower specificity (-1.3%) and PPV (-6.1%), and a higher recall rate (+ 1.3%) compared to first reader (p < 0.0001 for all), while Standalone AIspec had a lower sensitivity (-5.1%; p < 0.0001), PPV (-1.3%; p = 0.01) and NPV (-0.04%; p = 0.0002). Compared to combined reading, Integrated AIsens achieved higher sensitivity (+ 2.3%; p = 0.0004), but lower specificity (-0.6%) and PPV (-3.9%) as well as higher recall rate (+ 0.6%) and arbitration rate (+ 2.2%; p < 0.0001 for all). Integrated AIspec showed no significant difference in any outcome measures apart from a slightly higher arbitration rate (p < 0.0001). Subgroup analyses showed higher detection of interval cancers by Standalone AI and Integrated AI at both thresholds (p < 0.0001 for all) with a varying composition of detected cancers across multiple subgroups of tumour characteristics.ConclusionsReplacing first reader in double reading with an AI could be feasible but choosing an appropriate AI threshold is crucial to maintaining cancer detection accuracy and workload.
| S-EPMC10731688 | biostudies-literature
Project description:ObjectivesThis study surveyed the views of breast screening readers in the UK on how to incorporate Artificial Intelligence (AI) technology into breast screening mammography.MethodsAn online questionnaire was circulated to the UK breast screening readers. Questions included their degree of approval of four AI implementation scenarios: AI as triage, AI as a companion reader/reader aid, AI replacing one of the initial two readers, and AI replacing all readers. They were also asked to rank five AI representation options (discrete opinion; mammographic scoring; percentage score with 100% indicating malignancy; region of suspicion; heat map) and indicate which evidence they considered necessary to support the implementation of AI into their practice among six options offered.ResultsThe survey had 87 nationally accredited respondents across the UK; 73 completed the survey in full. Respondents approved of AI replacing one of the initial two human readers and objected to AI replacing all human readers. Participants were divided on AI as triage and AI as a reader companion. A region of suspicion superimposed on the image was the preferred AI representation option. Most screen readers considered national guidelines (77%), studies using a nationally representative dataset (65%) and independent prospective studies (60%) as essential evidence. Participants' free-text comments highlighted concerns and the need for additional validation.ConclusionsOverall, screen readers supported the introduction of AI as a partial replacement of human readers and preferred a graphical indication of the suspected tumour area, with further evidence and national guidelines considered crucial prior to implementation.
| S-EPMC9733732 | biostudies-literature
Project description:ImportancePrevious research has shown good discrimination of short-term risk using an artificial intelligence (AI) risk prediction model (Mirai). However, no studies have been undertaken to evaluate whether this might translate into economic gains.ObjectiveTo assess the cost-effectiveness of incorporating risk-stratified screening using a breast cancer AI model into the United Kingdom (UK) National Breast Cancer Screening Program.Design, setting, and participantsThis study, conducted from January 1, 2023, to January 31, 2024, involved the development of a decision analytical model to estimate health-related quality of life, cancer survival rates, and costs over the lifetime of the female population eligible for screening. The analysis took a UK payer perspective, and the simulated cohort consisted of women aged 50 to 70 years at screening.ExposuresMammography screening at 1 to 6 yearly screening intervals based on breast cancer risk and standard care (screening every 3 years).Main outcomes and measuresIncremental net monetary benefit based on quality-adjusted life-years (QALYs) and National Health Service (NHS) costs (given in pounds sterling; to convert to US dollars, multiply by 1.28).ResultsArtificial intelligence-based risk-stratified programs were estimated to be cost-saving and increase QALYs compared with the current screening program. A screening schedule of every 6 years for lowest-risk individuals, biannually and triennially for those below and above average risk, respectively, and annually for those at highest risk was estimated to give yearly net monetary benefits within the NHS of approximately £60.4 (US $77.3) million and £85.3 (US $109.2) million, with QALY values set at £20 000 (US $25 600) and £30 000 (US $38 400), respectively. Even in scenarios where decision-makers hesitate to allocate additional NHS resources toward screening, implementing the proposed strategies at a QALY value of £1 (US $1.28) was estimated to generate a yearly monetary benefit of approximately £10.6 (US $13.6) million.Conclusions and relevanceIn this decision analytical model study of integrating risk-stratified screening with a breast cancer AI model into the UK National Breast Cancer Screening Program, risk-stratified screening was likely to be cost-effective, yielding added health benefits at reduced costs. These results are particularly relevant for health care settings where resources are under pressure. New studies to prospectively evaluate AI-guided screening appear warranted.
| S-EPMC11377997 | biostudies-literature
Project description:Artificial intelligence (AI) readers of mammograms compare favourably to individual radiologists in detecting breast cancer. However, AI readers cannot perform at the level of multi-reader systems used by screening programs in countries such as Australia, Sweden, and the UK. Therefore, implementation demands human-AI collaboration. Here, we use a large, high-quality retrospective mammography dataset from Victoria, Australia to conduct detailed simulations of five potential AI-integrated screening pathways, and examine human-AI interaction effects to explore automation bias. Operating an AI reader as a second reader or as a high confidence filter improves current screening outcomes by 1.9-2.5% in sensitivity and up to 0.6% in specificity, achieving 4.6-10.9% reduction in assessments and 48-80.7% reduction in human reads. Automation bias degrades performance in multi-reader settings but improves it for single-readers. This study provides insight into feasible approaches for AI-integrated screening pathways and prospective studies necessary prior to clinical adoption.
| S-EPMC11364867 | biostudies-literature
Project description:Artificial intelligence (AI) algorithms have been retrospectively evaluated as replacement for one radiologist in screening mammography double-reading; however, methods for resolving discordance between radiologists and AI in the absence of 'real-world' arbitration may underestimate cancer detection rate (CDR) and recall. In 108,970 consecutive screens from a population screening program (BreastScreen WA, Western Australia), 20,120 were radiologist/AI discordant without real-world arbitration. Recall probabilities were randomly assigned for these screens in 1000 simulations. Recall thresholds for screen-detected and interval cancers (sensitivity) and no cancer (false-positive proportion, FPP) were varied to calculate mean CDR and recall rate for the entire cohort. Assuming 100% sensitivity, the maximum CDR was 7.30 per 1000 screens. To achieve >95% probability that the mean CDR exceeded the screening program CDR (6.97 per 1000), interval cancer sensitivities ≥63% (at 100% screen-detected sensitivity) and ≥91% (at 80% screen-detected sensitivity) were required. Mean recall rate was relatively constant across sensitivity assumptions, but varied by FPP. FPP > 6.5% resulted in recall rates that exceeded the program estimate (3.38%). CDR improvements depend on a majority of interval cancers being detected in radiologist/AI discordant screens. Such improvements are likely to increase recall, requiring careful monitoring where AI is deployed for screen-reading.
| S-EPMC11869508 | biostudies-literature
Project description:ObjectivesTo examine the various roles of radiologists in different steps of developing artificial intelligence (AI) applications.Materials and methodsThrough the case study of eight companies active in developing AI applications for radiology, in different regions (Europe, Asia, and North America), we conducted 17 semi-structured interviews and collected data from documents. Based on systematic thematic analysis, we identified various roles of radiologists. We describe how each role happens across the companies and what factors impact how and when these roles emerge.ResultsWe identified 9 roles that radiologists play in different steps of developing AI applications: (1) problem finder (in 4 companies); (2) problem shaper (in 3 companies); (3) problem dominator (in 1 company); (4) data researcher (in 2 companies); (5) data labeler (in 3 companies); (6) data quality controller (in 2 companies); (7) algorithm shaper (in 3 companies); (8) algorithm tester (in 6 companies); and (9) AI researcher (in 1 company).ConclusionsRadiologists can play a wide range of roles in the development of AI applications. How actively they are engaged and the way they are interacting with the development teams significantly vary across the cases. Radiologists need to become proactive in engaging in the development process and embrace new roles.Key points• Radiologists can play a wide range of roles during the development of AI applications. • Both radiologists and developers need to be open to new roles and ways of interacting during the development process. • The availability of resources, time, expertise, and trust are key factors that impact how actively radiologists play roles in the development process.
| S-EPMC8050223 | biostudies-literature
Project description:The present investigation has been conducted on one hundred tissue fragments of breast cancer, collected and immediately cryopreserved following the surgical resection. The specimens were selected from patients with invasive ductal carcinoma of the breast, the most frequent and potentially aggressive type of mammary cancer, with the objective to increase the knowledge of breast cancer molecular markers potentially useful for clinical applications. The proteomic screening; by 2D-IPG and mass spectrometry; allowed us to identify two main classes of protein clusters: proteins expressed ubiquitously at high levels in all patients; and proteins expressed sporadically among the same patients. Within the group of ubiquitous proteins, glycolytic enzymes and proteins with anti-apoptotic activity were predominant. Among the sporadic ones, proteins involved in cell motility, molecular chaperones and proteins involved in the detoxification appeared prevalent. The data of the present study indicates that the primary tumor growth is reasonably supported by concurrent events: the inhibition of apoptosis and stimulation of cellular proliferation, and the increased expression of glycolytic enzymes with multiple functions. The second phase of the evolution of the tumor can be prematurely scheduled by the occasional presence of proteins involved in cell motility and in the defenses of the oxidative stress. We suggest that this approach on large-scale 2D-IPG proteomics of breast cancer is currently a valid tool that offers the opportunity to evaluate on the same assay the presence and recurrence of individual proteins, their isoforms and short forms, to be proposed as prognostic indicators and susceptibility to metastasis in patients operated on for invasive ductal carcinoma of the breast.
| S-EPMC5620532 | biostudies-literature
Project description:Artificial intelligence (AI) improves the accuracy of mammography screening, but prospective evidence, particularly in a single-read setting, remains limited. This study compares the diagnostic accuracy of breast radiologists with and without AI-based computer-aided detection (AI-CAD) for screening mammograms in a real-world, single-read setting. A prospective multicenter cohort study is conducted within South Korea's national breast cancer screening program for women. The primary outcomes are screen-detected breast cancer within one year, with a focus on cancer detection rates (CDRs) and recall rates (RRs) of radiologists. A total of 24,543 women are included in the final cohort, with 140 (0.57%) screen-detected breast cancers. The CDR is significantly higher by 13.8% for breast radiologists using AI-CAD (n = 140 [5.70‰]) compared to those without AI (n = 123 [5.01‰]; p < 0.001), with no significant difference in RRs (p = 0.564). These preliminary results show a significant improvement in CDRs without affecting RRs in a radiologist's standard single-reading setting (ClinicalTrials.gov: NCT05024591).
| S-EPMC11885569 | biostudies-literature
Project description:BackgroundArtificial intelligence (AI) has been proposed to reduce false-positive screens, increase cancer detection rates (CDRs), and address resourcing challenges faced by breast screening programs. We compared the accuracy of AI versus radiologists in real-world population breast cancer screening, and estimated potential impacts on CDR, recall and workload for simulated AI-radiologist reading.MethodsExternal validation of a commercially-available AI algorithm in a retrospective cohort of 108,970 consecutive mammograms from a population-based screening program, with ascertained outcomes (including interval cancers by registry linkage). Area under the ROC curve (AUC), sensitivity and specificity for AI were compared with radiologists who interpreted the screens in practice. CDR and recall were estimated for simulated AI-radiologist reading (with arbitration) and compared with program metrics.FindingsThe AUC for AI was 0.83 compared with 0.93 for radiologists. At a prospective threshold, sensitivity for AI (0.67; 95% CI: 0.64-0.70) was comparable to radiologists (0.68; 95% CI: 0.66-0.71) with lower specificity (0.81 [95% CI: 0.81-0.81] versus 0.97 [95% CI: 0.97-0.97]). Recall rate for AI-radiologist reading (3.14%) was significantly lower than for the BSWA program (3.38%) (-0.25%; 95% CI: -0.31 to -0.18; P < 0.001). CDR was also lower (6.37 versus 6.97 per 1000) (-0.61; 95% CI: -0.77 to -0.44; P < 0.001); however, AI detected interval cancers that were not found by radiologists (0.72 per 1000; 95% CI: 0.57-0.90). AI-radiologist reading increased arbitration but decreased overall screen-reading volume by 41.4% (95% CI: 41.2-41.6).InterpretationReplacement of one radiologist by AI (with arbitration) resulted in lower recall and overall screen-reading volume. There was a small reduction in CDR for AI-radiologist reading. AI detected interval cases that were not identified by radiologists, suggesting potentially higher CDR if radiologists were unblinded to AI findings. These results indicate AI's potential role as a screen-reader of mammograms, but prospective trials are required to determine whether CDR could improve if AI detection was actioned in double-reading with arbitration.FundingNational Breast Cancer Foundation (NBCF), National Health and Medical Research Council (NHMRC).
| S-EPMC9996220 | biostudies-literature
Project description:Breast cancer is one of the most common invasive tumors causing high mortality among women. It is characterized by high heterogeneity regarding its biological and clinical characteristics. Several high-throughput assays have been used to collect genome-wide information for many patients in large collaborative studies. This knowledge has improved our understanding of its biology and led to new methods of diagnosing and treating the disease. In particular, system biology has become a valid approach to obtain better insights into breast cancer biological mechanisms. A crucial component of current research lies in identifying novel biomarkers that can be predictive for breast cancer patient prognosis on the basis of the molecular signature of the tumor sample. However, the high dimension and low sample size of data greatly increase the difficulty of cancer survival analysis demanding for the development of ad-hoc statistical methods. In this work, we propose novel screening-network methods that predict patient survival outcome by screening key survival-related genes and we assess the capability of the proposed approaches using METABRIC dataset. In particular, we first identify a subset of genes by using variable screening techniques on gene expression data. Then, we perform Cox regression analysis by incorporating network information associated with the selected subset of genes. The novelty of this work consists in the improved prediction of survival responses due to the different types of screenings (i.e., a biomedical-driven, data-driven and a combination of the two) before building the network-penalized model. Indeed, the combination of the two screening approaches allows us to use the available biological knowledge on breast cancer and complement it with additional information emerging from the data used for the analysis. Moreover, we also illustrate how to extend the proposed approaches to integrate an additional omic layer, such as copy number aberrations, and we show that such strategies can further improve our prediction capabilities. In conclusion, our approaches allow to discriminate patients in high-and low-risk groups using few potential biomarkers and therefore, can help clinicians to provide more precise prognoses and to facilitate the subsequent clinical management of patients at risk of disease.
| S-EPMC6011013 | biostudies-literature