Dataset Information

Deep Neural Frameworks Improve the Accuracy of General Practitioners in the Classification of Pigmented Skin Lesions.

ABSTRACT: This study evaluated whether deep learning frameworks trained in large datasets can help non-dermatologist physicians improve their accuracy in categorizing the seven most common pigmented skin lesions. Open-source skin images were downloaded from the International Skin Imaging Collaboration (ISIC) archive. Different deep neural networks (DNNs) (n = 8) were trained based on a random dataset constituted of 8015 images. A test set of 2003 images was used to assess the classifiers' performance at low (300 × 224 RGB) and high (600 × 450 RGB) image resolution and aggregated data (age, sex and lesion localization). We also organized two different contests to compare the DNN performance to that of general practitioners by means of unassisted image observation. Both at low and high image resolution, the DNN framework differentiated dermatological images with appreciable performance. In all cases, the accuracy was improved when adding clinical data to the framework. Finally, the least accurate DNN outperformed general practitioners. The physician's accuracy was statistically improved when allowed to use the output of this algorithmic framework as guidance. DNNs are proven to be high performers as skin lesion classifiers and can improve general practitioner diagnosis accuracy in a routine clinical scenario.

SUBMITTER: Lucius M

PROVIDER: S-EPMC7698907 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Deep Neural Frameworks Improve the Accuracy of General Practitioners in the Classification of Pigmented Skin Lesions.

Lucius Maximiliano M De All Jorge J De All José Antonio JA Belvisi Martín M Radizza Luciana L Lanfranconi Marisa M Lorenzatti Victoria V Galmarini Carlos M CM

Diagnostics (Basel, Switzerland) 20201118 11

This study evaluated whether deep learning frameworks trained in large datasets can help non-dermatologist physicians improve their accuracy in categorizing the seven most common pigmented skin lesions. Open-source skin images were downloaded from the International Skin Imaging Collaboration (ISIC) archive. Different deep neural networks (DNNs) (<i>n</i> = 8) were trained based on a random dataset constituted of 8015 images. A test set of 2003 images was used to assess the classifiers' performan ...[more]

PMID: 33218060

Similar Datasets

Project description:Recent studies have demonstrated the usefulness of convolutional neural networks (CNNs) to classify images of melanoma, with accuracies comparable to those achieved by dermatologists. However, the performance of a CNN trained with only clinical images of a pigmented skin lesion in a clinical image classification task, in competition with dermatologists, has not been reported to date. In this study, we extracted 5846 clinical images of pigmented skin lesions from 3551 patients. Pigmented skin lesions included malignant tumors (malignant melanoma and basal cell carcinoma) and benign tumors (nevus, seborrhoeic keratosis, senile lentigo, and hematoma/hemangioma). We created the test dataset by randomly selecting 666 patients out of them and picking one image per patient, and created the training dataset by giving bounding-box annotations to the rest of the images (4732 images, 2885 patients). Subsequently, we trained a faster, region-based CNN (FRCNN) with the training dataset and checked the performance of the model on the test dataset. In addition, ten board-certified dermatologists (BCDs) and ten dermatologic trainees (TRNs) took the same tests, and we compared their diagnostic accuracy with FRCNN. For six-class classification, the accuracy of FRCNN was 86.2%, and that of the BCDs and TRNs was 79.5% (p = 0.0081) and 75.1% (p < 0.00001), respectively. For two-class classification (benign or malignant), the accuracy, sensitivity, and specificity were 91.5%, 83.3%, and 94.5% by FRCNN; 86.6%, 86.3%, and 86.6% by BCD; and 85.3%, 83.5%, and 85.9% by TRN, respectively. False positive rates and positive predictive values were 5.5% and 84.7% by FRCNN, 13.4% and 70.5% by BCD, and 14.1% and 68.5% by TRN, respectively. We compared the classification performance of FRCNN with 20 dermatologists. As a result, the classification accuracy of FRCNN was better than that of the dermatologists. In the future, we plan to implement this system in society and have it used by the general public, in order to improve the prognosis of skin cancer.

Project description:1,25-dihydroxyvitamin D3 affects proliferation, differentiation, and apoptosis and protects DNA against oxidative damage with a net tumorostatic and anticarcinogenic effect. It acts through a specific nuclear receptor that is widely distributed through the body. Although a beneficial role of vitamin D in melanoma patients has been suggested, there is lack of information on the changes in the expression pattern of vitamin D receptor during progression of pigmented lesions. Using immunohistochemistry, we analyzed the expression of vitamin D receptor in 140 samples obtained form 82 patients, including 25 benign nevi, 70 primary cutaneous melanomas, 35 metastases, 5 re-excisions, and 5 normal skin biopsies. The strongest expression was observed in normal skin that significantly decreased in melanocytic proliferations with the following order of expression: normal skin > melanocytic nevi > melanomas = metastases. The vitamin D receptor expression in skin surrounding nevi and melanoma was also significantly reduced as compared to normal skin. Tumor-infiltrating and lymph node lymphocytes retained high levels of vitamin D receptor. There was negative correlation between tumor progression and vitamin D receptor expression with a remarkable decrease of the immunoreactivity in nuclei of melanoma cells at vertical versus radial growth phases and with metastatic melanomas showing the lowest cytoplasmic receptor staining. Furthermore, lack of the receptor expression in primary melanomas and metastases was related to shorter overall patients' survival. In addition, the receptor expression decreased in melanized melanoma cells in comparison to amelanotic or poorly pigmented cells. Therefore, we propose that reduction or absence of vitamin D receptor is linked to progression of melanocytic lesions, that its lack affects survival of melanoma patients, and that melanogenesis can attenuate receptor expression. In conclusion, changes in vitamin D receptor expression pattern can serve as important variables for diagnosis, predicting clinical outcome of the disease, and/or as a guidance for novel therapy of melanomas based on use of vitamin D or its derivatives.

Project description:BackgroundWhether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions.MethodsFor this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms.FindingsBetween Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06-7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9-12·9 vs 3·6%, 0·8-6·3; p<0·0001).InterpretationState-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research.FundingNone.

Project description:BACKGROUND:Although death certificates (DCs) provide valuable health information which may help to guide local health policies and priorities, there is little information concerning their validity in Thailand. First-year general practitioners (GPs) have a major role in DC completion, especially in provincial general hospitals. The aim of this study was to evaluate the accuracy and factors influencing the accuracy of DCs completed by first-year GPs in Thailand, compared with the cause of death (COD) derived from medical records by experts. METHODS:This retrospective study was conducted at 14 provincial general hospitals in Thailand during the June 2011 to May 2012 study period. Medical records and DCs completed by first-year GPs who graduated from 16 Thai medical schools were sampled. The cause of death recorded on the DCs was compared with the medical conditions and histories derived from patient medical records. A cross-sectional survey of the 210 GPs who completed the DCs reviewed in this study was also conducted. Respondent GPs' demographic characteristics, factors associated with COD, and COD coding system were evaluated. RESULTS:Five hundred and sixty-three medical records and corresponding DCs were included. Of those, 36.9% of DCs were found to be correct. Common mistakes included incorrect sequence of events leading to death (32.4%), and mode of death use (26.2%). Of the 210 GPs, 155 questionnaires were completed and returned. The mean time spent on recording COD and completing DCs in the medical school curriculum was 2.1?±?0.9 h and only 27.7% of participants had experience in completing DCs by themselves during medical school. Mean medical school GPA was significantly higher in the correctly completed DC GPs group than in the incorrectly completed DC GPs group (3.3?±?0.4 vs. 3.2?±?0.3; p?=?0.03). However, no significant difference was found for other factors associated with COD between groups. CONCLUSIONS:This is the first study documenting gaps and disparities in DC accuracy, and factors influencing completion of DCs among first-year GPs in Thailand, based on a clinical assessment of medical records. GPs made errors on 63.1% of DCs. This finding suggests that proven education, system-related support, and additional training interventions specific to DC completion are required.

Dataset Information

Deep Neural Frameworks Improve the Accuracy of General Practitioners in the Classification of Pigmented Skin Lesions.

Publications

Deep Neural Frameworks Improve the Accuracy of General Practitioners in the Classification of Pigmented Skin Lesions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets