Dataset Information

Human Versus Machine: Comparing a Deep Learning Algorithm to Human Gradings for Detecting Glaucoma on Fundus Photographs.

ABSTRACT:

Purpose

To compare the diagnostic performance of human gradings vs predictions provided by a machine-to-machine (M2M) deep learning (DL) algorithm trained to quantify retinal nerve fiber layer (RNFL) damage on fundus photographs.

Design

Evaluation of a machine learning algorithm.

Methods

An M2M DL algorithm trained with RNFL thickness parameters from spectral-domain optical coherence tomography was applied to a subset of 490 fundus photos of 490 eyes of 370 subjects graded by 2 glaucoma specialists for the probability of glaucomatous optical neuropathy (GON), and estimates of cup-to-disc (C/D) ratios. Spearman correlations with standard automated perimetry (SAP) global indices were compared between the human gradings vs the M2M DL-predicted RNFL thickness values. The area under the receiver operating characteristic curves (AUC) and partial AUC for the region of clinically meaningful specificity (85%-100%) were used to compare the ability of each output to discriminate eyes with repeatable glaucomatous SAP defects vs eyes with normal fields.

Results

The M2M DL-predicted RNFL thickness had a significantly stronger absolute correlation with SAP mean deviation (rho=0.54) than the probability of GON given by human graders (rho=0.48; P < .001). The partial AUC for the M2M DL algorithm was significantly higher than that for the probability of GON by human graders (partial AUC = 0.529 vs 0.411, respectively; P = .016).

Conclusion

An M2M DL algorithm performed as well as, if not better than, human graders at detecting eyes with repeatable glaucomatous visual field loss. This DL algorithm could potentially replace human graders in population screening efforts for glaucoma.

SUBMITTER: Jammal AA

PROVIDER: S-EPMC7073295 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Human Versus Machine: Comparing a Deep Learning Algorithm to Human Gradings for Detecting Glaucoma on Fundus Photographs.

Jammal Alessandro A AA Thompson Atalie C AC Mariottoni Eduardo B EB Berchuck Samuel I SI Urata Carla N CN Estrela Tais T Wakil Susan M SM Costa Vital P VP Medeiros Felipe A FA

American journal of ophthalmology 20191112

<h4>Purpose</h4>To compare the diagnostic performance of human gradings vs predictions provided by a machine-to-machine (M2M) deep learning (DL) algorithm trained to quantify retinal nerve fiber layer (RNFL) damage on fundus photographs.<h4>Design</h4>Evaluation of a machine learning algorithm.<h4>Methods</h4>An M2M DL algorithm trained with RNFL thickness parameters from spectral-domain optical coherence tomography was applied to a subset of 490 fundus photos of 490 eyes of 370 subjects graded ...[more]

PMID: 31730838

Similar Datasets

Project description:PurposePrevious approaches using deep learning (DL) algorithms to classify glaucomatous damage on fundus photographs have been limited by the requirement for human labeling of a reference training set. We propose a new approach using quantitative spectral-domain (SD) OCT data to train a DL algorithm to quantify glaucomatous structural damage on optic disc photographs.DesignCross-sectional study.ParticipantsA total of 32 820 pairs of optic disc photographs and SD OCT retinal nerve fiber layer (RNFL) scans from 2312 eyes of 1198 participants.MethodsThe sample was divided randomly into validation plus training (80%) and test (20%) sets, with randomization performed at the patient level. A DL convolutional neural network was trained to assess optic disc photographs and predict SD OCT average RNFL thickness.Main outcome measuresThe DL algorithm performance was evaluated in the test sample by evaluating correlation and agreement between the predictions and actual SD OCT measurements. We also assessed the ability to discriminate eyes with glaucomatous visual field loss from healthy eyes with area under the receiver operating characteristic (ROC) curves.ResultsThe mean prediction of average RNFL thickness from all 6292 optic disc photographs in the test set was 83.3±14.5 μm, whereas the mean average RNFL thickness from all corresponding SD OCT scans was 82.5±16.8 μm (P = 0.164). There was a very strong correlation between predicted and observed RNFL thickness values (Pearson r = 0.832; R2 = 69.3%; P < 0.001), with mean absolute error of the predictions of 7.39 μm. The area under the ROC curves for discriminating glaucomatous from healthy eyes with the DL predictions and actual SD OCT average RNFL thickness measurements were 0.944 (95% confidence interval [CI], 0.912-0.966) and 0.940 (95% CI, 0.902-0.966), respectively (P = 0.724).ConclusionsWe introduced a novel DL approach to assess fundus photographs and provide quantitative information about the amount of neural damage that can be used to diagnose and stage glaucoma. In addition, training neural networks to predict SD OCT data objectively represents a new approach that overcomes limitations of human labeling and could be useful in other areas of ophthalmology.

Project description:BackgroundTo verify efficacy of automatic screening and classification of glaucoma with deep learning system.MethodsA cross-sectional, retrospective study in a tertiary referral hospital. Patients with healthy optic disc, high-tension, or normal-tension glaucoma were enrolled. Complicated non-glaucomatous optic neuropathy was excluded. Colour and red-free fundus images were collected for development of DLS and comparison of their efficacy. The convolutional neural network with the pre-trained EfficientNet-b0 model was selected for machine learning. Glaucoma screening (Binary) and ternary classification with or without additional demographics (age, gender, high myopia) were evaluated, followed by creating confusion matrix and heatmaps. Area under receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and F1 score were viewed as main outcome measures.ResultsTwo hundred and twenty-two cases (421 eyes) were enrolled, with 1851 images in total (1207 normal and 644 glaucomatous disc). Train set and test set were comprised of 1539 and 312 images, respectively. If demographics were not provided, AUC, accuracy, precision, sensitivity, F1 score, and specificity of our deep learning system in eye-based glaucoma screening were 0.98, 0.91, 0.86, 0.86, 0.86, and 0.94 in test set. Same outcome measures in eye-based ternary classification without demographic data were 0.94, 0.87, 0.87, 0.87, 0.87, and 0.94 in our test set, respectively. Adding demographics has no significant impact on efficacy, but establishing a linkage between eyes and images is helpful for a better performance. Confusion matrix and heatmaps suggested that retinal lesions and quality of photographs could affect classification. Colour fundus images play a major role in glaucoma classification, compared to red-free fundus images.ConclusionsPromising results with high AUC and specificity were shown in distinguishing normal optic nerve from glaucomatous fundus images and doing further classification.

Project description:PURPOSE:To assess the utility of deep learning in the detection of geographic atrophy (GA) from color fundus photographs and to explore potential utility in detecting central GA (CGA). DESIGN:A deep learning model was developed to detect the presence of GA in color fundus photographs, and 2 additional models were developed to detect CGA in different scenarios. PARTICIPANTS:A total of 59?812 color fundus photographs from longitudinal follow-up of 4582 participants in the Age-Related Eye Disease Study (AREDS) dataset. Gold standard labels were from human expert reading center graders using a standardized protocol. METHODS:A deep learning model was trained to use color fundus photographs to predict GA presence from a population of eyes with no AMD to advanced AMD. A second model was trained to predict CGA presence from the same population. A third model was trained to predict CGA presence from the subset of eyes with GA. For training and testing, 5-fold cross-validation was used. For comparison with human clinician performance, model performance was compared with that of 88 retinal specialists. MAIN OUTCOME MEASURES:Area under the curve (AUC), accuracy, sensitivity, specificity, and precision. RESULTS:The deep learning models (GA detection, CGA detection from all eyes, and centrality detection from GA eyes) had AUCs of 0.933-0.976, 0.939-0.976, and 0.827-0.888, respectively. The GA detection model had accuracy, sensitivity, specificity, and precision of 0.965 (95% confidence interval [CI], 0.959-0.971), 0.692 (0.560-0.825), 0.978 (0.970-0.985), and 0.584 (0.491-0.676), respectively, compared with 0.975 (0.971-0.980), 0.588 (0.468-0.707), 0.982 (0.978-0.985), and 0.368 (0.230-0.505) for the retinal specialists. The CGA detection model had values of 0.966 (0.957-0.975), 0.763 (0.641-0.885), 0.971 (0.960-0.982), and 0.394 (0.341-0.448). The centrality detection model had values of 0.762 (0.725-0.799), 0.782 (0.618-0.945), 0.729 (0.543-0.916), and 0.799 (0.710-0.888). CONCLUSIONS:A deep learning model demonstrated high accuracy for the automated detection of GA. The AUC was noninferior to that of human retinal specialists. Deep learning approaches may also be applied to the identification of CGA. The code and pretrained models are publicly available at https://github.com/ncbi-nlp/DeepSeeNet.

Project description:ObjectiveTo evaluate the performance of a deep learning system (DLS) in classifying the severity of papilledema associated with increased intracranial pressure on standard retinal fundus photographs.MethodsA DLS was trained to automatically classify papilledema severity in 965 patients (2,103 mydriatic fundus photographs), representing a multiethnic cohort of patients with confirmed elevated intracranial pressure. Training was performed on 1,052 photographs with mild/moderate papilledema (MP) and 1,051 photographs with severe papilledema (SP) classified by a panel of experts. The performance of the DLS and that of 3 independent neuro-ophthalmologists were tested in 111 patients (214 photographs, 92 with MP and 122 with SP) by calculating the area under the receiver operating characteristics curve (AUC), accuracy, sensitivity, and specificity. Kappa agreement scores between the DLS and each of the 3 graders and among the 3 graders were calculated.ResultsThe DLS successfully discriminated between photographs of MP and SP, with an AUC of 0.93 (95% confidence interval [CI] 0.89-0.96) and an accuracy, sensitivity, and specificity of 87.9%, 91.8%, and 86.2%, respectively. This performance was comparable with that of the 3 neuro-ophthalmologists (84.1%, 91.8%, and 73.9%, p = 0.19, p = 1, p = 0.09, respectively). Misclassification by the DLS was mainly observed for moderate papilledema (Frisén grade 3). Agreement scores between the DLS and the neuro-ophthalmologists' evaluation was 0.62 (95% CI 0.57-0.68), whereas the intergrader agreement among the 3 neuro-ophthalmologists was 0.54 (95% CI 0.47-0.62).ConclusionsOur DLS accurately classified the severity of papilledema on an independent set of mydriatic fundus photographs, achieving a comparable performance with that of independent neuro-ophthalmologists.Classification of evidenceThis study provides Class II evidence that a DLS using mydriatic retinal fundus photographs accurately classified the severity of papilledema associated in patients with a diagnosis of increased intracranial pressure.

Dataset Information

Human Versus Machine: Comparing a Deep Learning Algorithm to Human Gradings for Detecting Glaucoma on Fundus Photographs.

Purpose

Design

Methods

Results

Conclusion

Publications

Human Versus Machine: Comparing a Deep Learning Algorithm to Human Gradings for Detecting Glaucoma on Fundus Photographs.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets