Dataset Information

Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes.

ABSTRACT: Importance:A deep learning system (DLS) is a machine learning technology with potential for screening diabetic retinopathy and related eye diseases. Objective:To evaluate the performance of a DLS in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, possible glaucoma, and age-related macular degeneration (AMD) in community and clinic-based multiethnic populations with diabetes. Design, Setting, and Participants:Diagnostic performance of a DLS for diabetic retinopathy and related eye diseases was evaluated using 494?661 retinal images. A DLS was trained for detecting diabetic retinopathy (using 76?370 images), possible glaucoma (125?189 images), and AMD (72?610 images), and performance of DLS was evaluated for detecting diabetic retinopathy (using 112?648 images), possible glaucoma (71?896 images), and AMD (35?948 images). Training of the DLS was completed in May 2016, and validation of the DLS was completed in May 2017 for detection of referable diabetic retinopathy (moderate nonproliferative diabetic retinopathy or worse) and vision-threatening diabetic retinopathy (severe nonproliferative diabetic retinopathy or worse) using a primary validation data set in the Singapore National Diabetic Retinopathy Screening Program and 10 multiethnic cohorts with diabetes. Exposures:Use of a deep learning system. Main Outcomes and Measures:Area under the receiver operating characteristic curve (AUC) and sensitivity and specificity of the DLS with professional graders (retinal specialists, general ophthalmologists, trained graders, or optometrists) as the reference standard. Results:In the primary validation dataset (n?=?14?880 patients; 71?896 images; mean [SD] age, 60.2 [2.2] years; 54.6% men), the prevalence of referable diabetic retinopathy was 3.0%; vision-threatening diabetic retinopathy, 0.6%; possible glaucoma, 0.1%; and AMD, 2.5%. The AUC of the DLS for referable diabetic retinopathy was 0.936 (95% CI, 0.925-0.943), sensitivity was 90.5% (95% CI, 87.3%-93.0%), and specificity was 91.6% (95% CI, 91.0%-92.2%). For vision-threatening diabetic retinopathy, AUC was 0.958 (95% CI, 0.956-0.961), sensitivity was 100% (95% CI, 94.1%-100.0%), and specificity was 91.1% (95% CI, 90.7%-91.4%). For possible glaucoma, AUC was 0.942 (95% CI, 0.929-0.954), sensitivity was 96.4% (95% CI, 81.7%-99.9%), and specificity was 87.2% (95% CI, 86.8%-87.5%). For AMD, AUC was 0.931 (95% CI, 0.928-0.935), sensitivity was 93.2% (95% CI, 91.1%-99.8%), and specificity was 88.7% (95% CI, 88.3%-89.0%). For referable diabetic retinopathy in the 10 additional datasets, AUC range was 0.889 to 0.983 (n?=?40?752 images). Conclusions and Relevance:In this evaluation of retinal images from multiethnic cohorts of patients with diabetes, the DLS had high sensitivity and specificity for identifying diabetic retinopathy and related eye diseases. Further research is necessary to evaluate the applicability of the DLS in health care settings and the utility of the DLS to improve vision outcomes.

SUBMITTER: Ting DSW

PROVIDER: S-EPMC5820739 | biostudies-literature | 2017 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes.

Ting Daniel Shu Wei DSW Cheung Carol Yim-Lui CY Lim Gilbert G Tan Gavin Siew Wei GSW Quang Nguyen D ND Gan Alfred A Hamzah Haslina H Garcia-Franco Renata R San Yeo Ian Yew IY Lee Shu Yen SY Wong Edmund Yick Mun EYM Sabanayagam Charumathi C Baskaran Mani M Ibrahim Farah F Tan Ngiap Chuan NC Finkelstein Eric A EA Lamoureux Ecosse L EL Wong Ian Y IY Bressler Neil M NM Sivaprasad Sobha S Varma Rohit R Jonas Jost B JB He Ming Guang MG Cheng Ching-Yu CY Cheung Gemmy Chui Ming GCM Aung Tin T Hsu Wynne W Lee Mong Li ML Wong Tien Yin TY

JAMA 20171201 22

<h4>Importance</h4>A deep learning system (DLS) is a machine learning technology with potential for screening diabetic retinopathy and related eye diseases.<h4>Objective</h4>To evaluate the performance of a DLS in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, possible glaucoma, and age-related macular degeneration (AMD) in community and clinic-based multiethnic populations with diabetes.<h4>Design, setting, and participants</h4>Diagnostic performance of a DLS ...[more]

PMID: 29234807

Similar Datasets

Project description:Importance:Quantitative measurements based on optical coherence tomographic angiography (OCTA) may have value in managing diabetic retinopathy (DR), but there is limited information on the ability of OCTA to distinguish eyes with DR. Objective:To evaluate the ability of measurements of retinal microvasculature using OCTA to distinguish healthy eyes from eyes with DR. Design, Setting, and Participants:In this prospective cross-sectional study, OCTA was used to examine the eyes of participants with type 2 diabetes with or without DR and the eyes of participants without diabetes from September 17, 2015, to April 6, 2016. Density maps based on superficial retinal layer (SRL) and deeper retinal layer (DRL) images were generated after a method to remove decorrelation tails was applied to the DRL images. Exposures:Both eyes of each participant were examined by means of a 3-mm OCTA scan and 7-field fundus photography using the Diabetic Retinopathy Severity Scale. Main Outcomes and Measures:Two measures were examined: perfusion density, based on the area of vessels, and vessel density, based on a map with vessels of 1-pixel width. The size of the foveal avascular zone was also calculated automatically, and so was the area under the receiver operating characteristic curve. Results:A total of 50 eyes from 26 participants with diabetes (10 women and 16 men; mean [SD] age, 64.9 [7.5] years) and 50 healthy eyes from 25 participants without diabetes (14 women and 11 men; mean [SD] age, 64.0 [7.1] years) were imaged. All participants were white. Vessel density measured in the SRL had the highest area under the receiver operating characteristic curve (0.893 [95% CI, 0.827-0.959]), compared with perfusion density in the SRL (0.794 [95% CI, 0.707-0.881]), foveal avascular zone area (0.472 [95% CI, 0.356-0.588]), and vessel density in the DRL (0.703 [95% CI, 0.601-0.805]). Vessel density in the SRL negatively correlated with best-corrected visual acuity (r?=?-0.28; P?=?.05) and severity of DR (r?=?-0.46; P?=?.001). Density metrics correlated with age. No correlation was detected between vascular density or foveal avascular zone metrics and hemoglobin A1C or duration of diabetes. Conclusions and Relevance:Vessel density measured by OCTA provides a quantitative metric of capillary closure that correlates with severity of DR and may allow staging, diagnosis, and monitoring that do not require subjective evaluation of fundus images.

Project description:ImportanceA retinopathy of prematurity (ROP) diagnosis currently relies on indirect ophthalmoscopy assessed by experienced ophthalmologists. A deep learning algorithm based on retinal images may facilitate early detection and timely treatment of ROP to improve visual outcomes.ObjectiveTo develop a retinal image-based, multidimensional, automated, deep learning platform for ROP screening and validate its performance accuracy.Design, setting, and participantsA total of 14 108 eyes of 8652 preterm infants who received ROP screening from 4 centers from November 4, 2010, to November 14, 2019, were included, and a total of 52 249 retinal images were randomly split into training, validation, and test sets. Four main dimensional independent classifiers were developed, including image quality, any stage of ROP, intraocular hemorrhage, and preplus/plus disease. Referral-warranted ROP was automatically generated by integrating the results of 4 classifiers at the image, eye, and patient levels. DeepSHAP, a method based on DeepLIFT and Shapley values (solution concepts in cooperative game theory), was adopted as the heat map technology to explain the predictions. The performance of the platform was further validated as compared with that of the experienced ROP experts. Data were analyzed from February 12, 2020, to June 24, 2020.ExposureA deep learning algorithm.Main outcomes and measuresThe performance of each classifier included true negative, false positive, false negative, true positive, F1 score, sensitivity, specificity, receiver operating characteristic, area under curve (AUC), and Cohen unweighted κ.ResultsA total of 14 108 eyes of 8652 preterm infants (mean [SD] gestational age, 32.9 [3.1] weeks; 4818 boys [60.4%] of 7973 with known sex) received ROP screening. The performance of all classifiers achieved an F1 score of 0.718 to 0.981, a sensitivity of 0.918 to 0.982, a specificity of 0.949 to 0.992, and an AUC of 0.983 to 0.998, whereas that of the referral system achieved an F1 score of 0.898 to 0.956, a sensitivity of 0.981 to 0.986, a specificity of 0.939 to 0.974, and an AUC of 0.9901 to 0.9956. Fine-grained and class-discriminative heat maps were generated by DeepSHAP in real time. The platform achieved a Cohen unweighted κ of 0.86 to 0.98 compared with a Cohen κ of 0.93 to 0.98 by the ROP experts.Conclusions and relevanceIn this diagnostic study, an automated ROP screening platform was able to identify and classify multidimensional pathologic lesions in the retinal images. This platform may be able to assist routine ROP screening in general and children hospitals.

Project description:Diabetic Retinopathy (DR) may result in various degrees of vision loss and even blindness if not diagnosed in a timely manner. Therefore, having an annual eye exam helps early detection to prevent vision loss in earlier stages, especially for diabetic patients. Recent technological advances made smartphone-based retinal imaging systems available on the market to perform small-sized, low-powered, and affordable DR screening in diverse environments. However, the accuracy of DR detection depends on the field of view and image quality. Since smartphone-based retinal imaging systems have much more compact designs than a traditional fundus camera, captured images are likely to be the low quality with a smaller field of view. Our motivation in this paper is to develop an automatic DR detection model for smartphone-based retinal images using the deep learning approach with the ResNet50 network. This study first utilized the well-known AlexNet, GoogLeNet, and ResNet50 architectures, using the transfer learning approach. Second, these frameworks were retrained with retina images from several datasets including EyePACS, Messidor, IDRiD, and Messidor-2 to investigate the effect of using images from the single, cross, and multiple datasets. Third, the proposed ResNet50 model is applied to smartphone-based synthetic images to explore the DR detection accuracy of smartphone-based retinal imaging systems. Based on the vision-threatening diabetic retinopathy detection results, the proposed approach achieved a high classification accuracy of 98.6%, with a 98.2% sensitivity and a 99.1% specificity while its AUC was 0.9978 on the independent test dataset. As the main contributions, DR detection accuracy was improved using the deep transfer learning approach for the ResNet50 network with publicly available datasets and the effect of the field of view in smartphone-based retinal imaging was studied. Although a smaller number of images were used in the training set compared with the existing studies, considerably acceptable high accuracies for validation and testing data were obtained.

Project description:ImportanceRecent studies have demonstrated the successful application of artificial intelligence (AI) for automated retinal disease diagnostics but have not addressed a fundamental challenge for deep learning systems: the current need for large, criterion standard-annotated retinal data sets for training. Low-shot learning algorithms, aiming to learn from a relatively low number of training data, may be beneficial for clinical situations involving rare retinal diseases or when addressing potential bias resulting from data that may not adequately represent certain groups for training, such as individuals older than 85 years.ObjectiveTo evaluate whether low-shot deep learning methods are beneficial when using small training data sets for automated retinal diagnostics.Design, setting, and participantsThis cross-sectional study, conducted from July 1, 2019, to June 21, 2020, compared different diabetic retinopathy classification algorithms, traditional and low-shot, for 2-class designations (diabetic retinopathy warranting referral vs not warranting referral). The public domain EyePACS data set was used, which originally included 88 692 fundi from 44 346 individuals. Statistical analysis was performed from February 1 to June 21, 2020.Main outcomes and measuresThe performance (95% CIs) of the various AI algorithms was measured via receiver operating curves and their area under the curve (AUC), precision recall curves, accuracy, and F1 score, evaluated for different training data sizes, ranging from 5120 to 10 samples per class.ResultsDeep learning algorithms, when trained with sufficiently large data sets (5120 samples per class), yielded comparable performance, with an AUC of 0.8330 (95% CI, 0.8140-0.8520) for a traditional approach (eg, fined-tuned ResNet), compared with low-shot methods (AUC, 0.8348 [95% CI, 0.8159-0.8537]) (using self-supervised Deep InfoMax [our method denoted as DIM]). However, when far fewer training images were available (n = 160), the traditional deep learning approach had an AUC decreasing to 0.6585 (95% CI, 0.6332-0.6838) and was outperformed by a low-shot method using self-supervision with an AUC of 0.7467 (95% CI, 0.7239-0.7695). At very low shots (n = 10), the traditional approach had performance close to chance, with an AUC of 0.5178 (95% CI, 0.4909-0.5447) compared with the best low-shot method (AUC, 0.5778 [95% CI, 0.5512-0.6044]).Conclusions and relevanceThese findings suggest the potential benefits of using low-shot methods for AI retinal diagnostics when a limited number of annotated training retinal images are available (eg, with rare ophthalmic diseases or when addressing potential AI bias).

Dataset Information

Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes.

Publications

Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets