Dataset Information

Artificial Intelligence and Its Effect on Dermatologists' Accuracy in Dermoscopic Melanoma Image Classification: Web-Based Survey Study.

ABSTRACT: BACKGROUND:Early detection of melanoma can be lifesaving but this remains a challenge. Recent diagnostic studies have revealed the superiority of artificial intelligence (AI) in classifying dermoscopic images of melanoma and nevi, concluding that these algorithms should assist a dermatologist's diagnoses. OBJECTIVE:The aim of this study was to investigate whether AI support improves the accuracy and overall diagnostic performance of dermatologists in the dichotomous image-based discrimination between melanoma and nevus. METHODS:Twelve board-certified dermatologists were presented disjoint sets of 100 unique dermoscopic images of melanomas and nevi (total of 1200 unique images), and they had to classify the images based on personal experience alone (part I) and with the support of a trained convolutional neural network (CNN, part II). Additionally, dermatologists were asked to rate their confidence in their final decision for each image. RESULTS:While the mean specificity of the dermatologists based on personal experience alone remained almost unchanged (70.6% vs 72.4%; P=.54) with AI support, the mean sensitivity and mean accuracy increased significantly (59.4% vs 74.6%; P=.003 and 65.0% vs 73.6%; P=.002, respectively) with AI support. Out of the 10% (10/94; 95% CI 8.4%-11.8%) of cases where dermatologists were correct and AI was incorrect, dermatologists on average changed to the incorrect answer for 39% (4/10; 95% CI 23.2%-55.6%) of cases. When dermatologists were incorrect and AI was correct (25/94, 27%; 95% CI 24.0%-30.1%), dermatologists changed their answers to the correct answer for 46% (11/25; 95% CI 33.1%-58.4%) of cases. Additionally, the dermatologists' average confidence in their decisions increased when the CNN confirmed their decision and decreased when the CNN disagreed, even when the dermatologists were correct. Reported values are based on the mean of all participants. Whenever absolute values are shown, the denominator and numerator are approximations as every dermatologist ended up rating a varying number of images due to a quality control step. CONCLUSIONS:The findings of our study show that AI support can improve the overall accuracy of the dermatologists in the dichotomous image-based discrimination between melanoma and nevus. This supports the argument for AI-based tools to aid clinicians in skin lesion classification and provides a rationale for studies of such classifiers in real-life settings, wherein clinicians can integrate additional information such as patient age and medical history into their decisions.

SUBMITTER: Maron RC

PROVIDER: S-EPMC7519424 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Artificial Intelligence and Its Effect on Dermatologists' Accuracy in Dermoscopic Melanoma Image Classification: Web-Based Survey Study.

Maron Roman C RC Utikal Jochen S JS Hekler Achim A Hauschild Axel A Sattler Elke E Sondermann Wiebke W Haferkamp Sebastian S Schilling Bastian B Heppt Markus V MV Jansen Philipp P Reinholz Markus M Franklin Cindy C Schmitt Laurenz L Hartmann Daniela D Krieghoff-Henning Eva E Schmitt Max M Weichenthal Michael M von Kalle Christof C Fröhling Stefan S Brinker Titus J TJ

Journal of medical Internet research 20200911 9

<h4>Background</h4>Early detection of melanoma can be lifesaving but this remains a challenge. Recent diagnostic studies have revealed the superiority of artificial intelligence (AI) in classifying dermoscopic images of melanoma and nevi, concluding that these algorithms should assist a dermatologist's diagnoses.<h4>Objective</h4>The aim of this study was to investigate whether AI support improves the accuracy and overall diagnostic performance of dermatologists in the dichotomous image-based di ...[more]

PMID: 32915161

Similar Datasets

Project description:BackgroundArtificial intelligence (AI) has become a powerful tool and is attracting more attention in the field of medicine. There are a number of AI studies focusing on skin diseases, and there are many AI products that have been applied in dermatology. However, the attitudes of dermatologists, specifically those from China, towards AI, is not clear as few, if any studies have focused on this issue.MethodsA web-based questionnaire was designed by experts from the Chinese Skin Image Database (CSID) and published on the UMER Doctor platform (an online learning platform for dermatologists developed by the Shanghai Wheat Color Intelligent Technology Company, China). A total of 1,228 Chinese dermatologists were recruited and provided answers to the questionnaire online. The differences of dermatologists' attitudes towards AI among the different groups (stratified by age, gender, hospital level, education degree, professional title, and hospital ownership) were compared by using the Mann-Whitney U test and the Kruskal-Wallis H test. The correlations between stratified factors and dermatologists' attitudes towards AI were calculated by using the Spearman's rank correlation test. SPSS (version 22.0) was utilized for all analyses. A two-sided P value <0.05 was considered statistically significant in all analyses.ResultsA total of 1,228 Chinese dermatologists from 30 provinces, autonomous regions, municipalities, and other regions (including Hong Kong, Macau, and Taiwan) participated in this survey. The dermatologists who participated acquired AI-related information mainly through the Internet, meetings or forums, and 70.51% of participated dermatologists acquired AI-related information by two or more approaches. In total, 99.51% of participated dermatologists pay attention (general, passive-active, and active attention) to information pertaining to AI. Stratified analyses revealed statistically significant differences in their attention levels (unconcerned, general, passive-active, and active attention) to AI-related information by gender, hospital level, education degree, and professional title (P values ≤1.79E-02). In total, 95.36% of the participated dermatologists thought the role of AI to be in "assisting the daily diagnosis and treatment activities for dermatologists". Stratified analyses about the thought of AI roles (unconcerned, useless, assist, and replace) showed that there was no statistically significant difference except for the hospital level (P value =4.09E-03). The correlations between stratified factors with attention levels and the opinions of AI roles showed extremely weak correlations. Furthermore, 64.17% of participated dermatologists thought secondary hospitals in China are in most need of the application AI, and 91.78% of participated dermatologists thought the priority implementation of AI should be in skin tumors.ConclusionsThe majority of Chinese dermatologists are interested in AI information and acquired information about AI through a variety of approaches. Nearly all dermatologists are attentive to information on AI and think the role of AI is in "assisting the daily diagnosis and treatment activities for dermatologists". Future AI implementation should be primarily focused on skin tumors and utilized in in secondary hospitals.

Project description:PurposeTo evaluate the diagnostic performance of image-based artificial intelligence (AI) studies in predicting muscle-invasive bladder cancer (MIBC). (2) To assess the reporting quality and methodological quality of these studies by Checklist for Artificial Intelligence in Medical Imaging (CLAIM), Radiomics Quality Score (RQS), and Prediction model Risk of Bias Assessment Tool (PROBAST).Materials and methodsWe searched Medline, Embase, Web of Science, and The Cochrane Library databases up to October 30, 2023. The eligible studies were evaluated using CLAIM, RQS, and PROBAST. Pooled sensitivity, specificity, and the diagnostic performances of these models for MIBC were also calculated.ResultsTwenty-one studies containing 4256 patients were included, of which 17 studies were employed for the quantitative statistical analysis. The CLAIM study adherence rate ranged from 52.5% to 75%, with a median of 64.1%. The RQS points of each study ranged from 2.78% to 50% points, with a median of 30.56% points. All models were rated as high overall ROB. The pooled area under the curve was 0.85 (95% confidence interval (CI) 0.81-0.88) for computed tomography, 0.92 (95% CI 0.89-0.94) for MRI, 0.89 (95% CI 0.86-0.92) for radiomics and 0.91 (95% CI 0.88-0.93) for deep learning, respectively.ConclusionAlthough AI-powered muscle-invasive bladder cancer-predictive models showed promising performance in the meta-analysis, the reporting quality and the methodological quality were generally low, with a high risk of bias.Critical relevance statementArtificial intelligence might improve the management of patients with bladder cancer. Multiple models for muscle-invasive bladder cancer prediction were developed. Quality assessment is needed to promote clinical application.Key pointsImage-based artificial intelligence models could aid in the identification of muscle-invasive bladder cancer. Current studies had low reporting quality, low methodological quality, and a high risk of bias. Future studies could focus on larger sample sizes and more transparent reporting of pathological evaluation, model explanation, and failure and sensitivity analyses.

Project description:BACKGROUND. Potential approaches for abbreviated knee MRI, including prospective acceleration with deep learning, have achieved limited clinical implementation. OBJECTIVE. The objective of this study was to evaluate the interreader agreement between conventional knee MRI and a 5-minute 3D quantitative double-echo steady-state (qDESS) sequence with automatic T2 mapping and deep learning super-resolutionaugmentation and to compare the diagnostic performance of the two methods regarding findings from arthroscopic surgery. METHODS. Fifty-one patients with knee pain underwent knee MRI that included an additional 3D qDESS sequence with automatic T2 mapping. Fourier interpolation was followed by prospective deep learning super resolution to enhance qDESS slice resolution twofold. A musculoskeletal radiologist and a radiology resident performed independent retrospective evaluations of articular cartilage, menisci, ligaments, bones, extensor mechanism, and synovium using conventional MRI. Following a 2-month washout period, readers reviewed qDESS images alone followed by qDESS with the automatic T2 maps. Interreader agreement between conventional MRI and qDESS was computed using percentage agreement and Cohen kappa. The sensitivity and specificity of conventional MRI, qDESS alone, and qDESS plus T2 mapping were compared with arthroscopic findings using exact McNemar tests. RESULTS. Conventional MRI and qDESS showed 92% agreement in evaluating all tissues. Kappa was 0.79 (95% CI, 0.76-0.81) across all imaging findings. In 43 patients who underwent arthroscopy, sensitivity and specificity were not significantly different (p = .23 to > .99) between conventional MRI (sensitivity, 58-93%; specificity, 27-87%) and qDESS alone (sensitivity, 54-90%; specificity, 23-91%) for cartilage, menisci, ligaments, and synovium. For grade 1 cartilage lesions, sensitivity and specificity were 33% and 56%, respectively, for conventional MRI; 23% and 53% for qDESS (p = .81); and 46% and 39% for qDESS with T2 mapping (p = .80). For grade 2A lesions, values were 27% and 53% for conventional MRI, 26% and 52% for qDESS (p = .02), and 58% and 40% for qDESS with T2 mapping (p < .001). CONCLUSION. The qDESS method prospectively augmented with deep learning showed strong interreader agreement with conventional knee MRI and near-equivalent diagnostic performance regarding arthroscopy. The ability of qDESS to automatically generate T2 maps increases sensitivity for cartilage abnormalities. CLINICAL IMPACT. Using prospective artificial intelligence to enhance qDESS image quality may facilitate an abbreviated knee MRI protocol while generating quantitative T2 maps.

Project description:ObjectiveTo examine the accuracy of artificial intelligence (AI) for the detection of breast cancer in mammography screening practice.DesignSystematic review of test accuracy studies.Data sourcesMedline, Embase, Web of Science, and Cochrane Database of Systematic Reviews from 1 January 2010 to 17 May 2021.Eligibility criteriaStudies reporting test accuracy of AI algorithms, alone or in combination with radiologists, to detect cancer in women's digital mammograms in screening practice, or in test sets. Reference standard was biopsy with histology or follow-up (for screen negative women). Outcomes included test accuracy and cancer type detected.Study selection and synthesisTwo reviewers independently assessed articles for inclusion and assessed the methodological quality of included studies using the QUality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool. A single reviewer extracted data, which were checked by a second reviewer. Narrative data synthesis was performed.ResultsTwelve studies totalling 131 822 screened women were included. No prospective studies measuring test accuracy of AI in screening practice were found. Studies were of poor methodological quality. Three retrospective studies compared AI systems with the clinical decisions of the original radiologist, including 79 910 women, of whom 1878 had screen detected cancer or interval cancer within 12 months of screening. Thirty four (94%) of 36 AI systems evaluated in these studies were less accurate than a single radiologist, and all were less accurate than consensus of two or more radiologists. Five smaller studies (1086 women, 520 cancers) at high risk of bias and low generalisability to the clinical context reported that all five evaluated AI systems (as standalone to replace radiologist or as a reader aid) were more accurate than a single radiologist reading a test set in the laboratory. In three studies, AI used for triage screened out 53%, 45%, and 50% of women at low risk but also 10%, 4%, and 0% of cancers detected by radiologists.ConclusionsCurrent evidence for AI does not yet allow judgement of its accuracy in breast cancer screening programmes, and it is unclear where on the clinical pathway AI might be of most benefit. AI systems are not sufficiently specific to replace radiologist double reading in screening programmes. Promising results in smaller studies are not replicated in larger studies. Prospective studies are required to measure the effect of AI in clinical practice. Such studies will require clear stopping rules to ensure that AI does not reduce programme specificity.Study registrationProtocol registered as PROSPERO CRD42020213590.

Dataset Information

Artificial Intelligence and Its Effect on Dermatologists' Accuracy in Dermoscopic Melanoma Image Classification: Web-Based Survey Study.

Publications

Artificial Intelligence and Its Effect on Dermatologists' Accuracy in Dermoscopic Melanoma Image Classification: Web-Based Survey Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets