Dataset Information

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms.

ABSTRACT: Purpose:To compare performance of independently developed deep learning algorithms for detecting glaucoma from fundus photographs and to evaluate strategies for incorporating new data into models. Methods:Two fundus photograph datasets from the Diagnostic Innovations in Glaucoma Study/African Descent and Glaucoma Evaluation Study and Matsue Red Cross Hospital were used to independently develop deep learning algorithms for detection of glaucoma at the University of California, San Diego, and the University of Tokyo. We compared three versions of the University of California, San Diego, and University of Tokyo models: original (no retraining), sequential (retraining only on new data), and combined (training on combined data). Independent datasets were used to test the algorithms. Results:The original University of California, San Diego and University of Tokyo models performed similarly (area under the receiver operating characteristic curve = 0.96 and 0.97, respectively) for detection of glaucoma in the Matsue Red Cross Hospital dataset, but not the Diagnostic Innovations in Glaucoma Study/African Descent and Glaucoma Evaluation Study data (0.79 and 0.92; P < .001), respectively. Model performance was higher when classifying moderate-to-severe compared with mild disease (area under the receiver operating characteristic curve = 0.98 and 0.91; P < .001), respectively. Models trained with the combined strategy generally had better performance across all datasets than the original strategy. Conclusions:Deep learning glaucoma detection can achieve high accuracy across diverse datasets with appropriate training strategies. Because model performance was influenced by the severity of disease, labeling, training strategies, and population characteristics, reporting accuracy stratified by relevant covariates is important for cross study comparisons. Translational Relevance:High sensitivity and specificity of deep learning algorithms for moderate-to-severe glaucoma across diverse populations suggest a role for artificial intelligence in the detection of glaucoma in primary care.

SUBMITTER: Christopher M

PROVIDER: S-EPMC7396194 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms.

Christopher Mark M Nakahara Kenichi K Bowd Christopher C Proudfoot James A JA Belghith Akram A Goldbaum Michael H MH Rezapour Jasmin J Weinreb Robert N RN Fazio Massimo A MA Girkin Christopher A CA Liebmann Jeffrey M JM De Moraes Gustavo G Murata Hiroshi H Tokumo Kana K Shibata Naoto N Fujino Yuri Y Matsuura Masato M Kiuchi Yoshiaki Y Tanito Masaki M Asaoka Ryo R Zangwill Linda M LM

Translational vision science & technology 20200428 2

<h4>Purpose</h4>To compare performance of independently developed deep learning algorithms for detecting glaucoma from fundus photographs and to evaluate strategies for incorporating new data into models.<h4>Methods</h4>Two fundus photograph datasets from the Diagnostic Innovations in Glaucoma Study/African Descent and Glaucoma Evaluation Study and Matsue Red Cross Hospital were used to independently develop deep learning algorithms for detection of glaucoma at the University of California, San ...[more]

PMID: 32818088

Similar Datasets

Project description:ImportanceAutomated deep learning (DL) analyses of fundus photographs potentially can reduce the cost and improve the efficiency of reading center assessment of end points in clinical trials.ObjectiveTo investigate the diagnostic accuracy of DL algorithms trained on fundus photographs from the Ocular Hypertension Treatment Study (OHTS) to detect primary open-angle glaucoma (POAG).Design, setting, and participantsIn this diagnostic study, 1636 OHTS participants from 22 sites with a mean (range) follow-up of 10.7 (0-14.3) years. A total of 66 715 photographs from 3272 eyes were used to train and test a ResNet-50 model to detect the OHTS Endpoint Committee POAG determination based on optic disc (287 eyes, 3502 photographs) and/or visual field (198 eyes, 2300 visual fields) changes. Three independent test sets were used to evaluate the generalizability of the model.Main outcomes and measuresAreas under the receiver operating characteristic curve (AUROC) and sensitivities at fixed specificities were calculated to compare model performance. Evaluation of false-positive rates was used to determine whether the DL model detected POAG before the OHTS Endpoint Committee POAG determination.ResultsA total of 1147 participants were included in the training set (661 [57.6%] female; mean age, 57.2 years; 95% CI, 56.6-57.8), 167 in the validation set (97 [58.1%] female; mean age, 57.1 years; 95% CI, 55.6-58.7), and 322 in the test set (173 [53.7%] female; mean age, 57.2 years; 95% CI, 56.1-58.2). The DL model achieved an AUROC of 0.88 (95% CI, 0.82-0.92) for the OHTS Endpoint Committee determination of optic disc or VF changes. For the OHTS end points based on optic disc changes or visual field changes, AUROCs were 0.91 (95% CI, 0.88-0.94) and 0.86 (95% CI, 0.76-0.93), respectively. False-positive rates (at 90% specificity) were higher in photographs of eyes that later developed POAG by disc or visual field (27.5% [56 of 204]) compared with eyes that did not develop POAG (11.4% [50 of 440]) during follow-up. The diagnostic accuracy of the DL model developed on the optic disc end point applied to 3 independent data sets was lower, with AUROCs ranging from 0.74 (95% CI, 0.70-0.77) to 0.79 (95% CI, 0.78-0.81).Conclusions and relevanceThe model's high diagnostic accuracy using OHTS photographs suggests that DL has the potential to standardize and automate POAG determination for clinical trials and management. In addition, the higher false-positive rate in early photographs of eyes that later developed POAG suggests that DL models detected POAG in some eyes earlier than the OHTS Endpoint Committee, reflecting the OHTS design that emphasized a high specificity for POAG determination by requiring a clinically significant change from baseline.

Project description:Background and Purpose- Selection of patients with acute ischemic stroke for endovascular treatment generally relies on dynamic susceptibility contrast magnetic resonance imaging or computed tomography perfusion. Dynamic susceptibility contrast magnetic resonance imaging requires injection of contrast, whereas computed tomography perfusion requires high doses of ionizing radiation. The purpose of this work was to develop and evaluate a deep learning (DL)-based algorithm for assisting the selection of suitable patients with acute ischemic stroke for endovascular treatment based on 3-dimensional pseudo-continuous arterial spin labeling (pCASL). Methods- A total of 167 image sets of 3-dimensional pCASL data from 137 patients with acute ischemic stroke scanned on 1.5T and 3.0T Siemens MR systems were included for neural network training. The concurrently acquired dynamic susceptibility contrast magnetic resonance imaging was used to produce labels of hypoperfused brain regions, analyzed using commercial software. The DL and 6 machine learning (ML) algorithms were trained with 10-fold cross-validation. The eligibility for endovascular treatment was determined retrospectively based on the criteria of perfusion/diffusion mismatch in the DEFUSE 3 trial (Endovascular Therapy Following Imaging Evaluation for Ischemic Stroke). The trained DL algorithm was further applied on twelve 3-dimensional pCASL data sets acquired on 1.5T and 3T General Electric MR systems, without fine-tuning of parameters. Results- The DL algorithm can predict the dynamic susceptibility contrast-defined hypoperfusion region in pCASL with a voxel-wise area under the curve of 0.958, while the 6 ML algorithms ranged from 0.897 to 0.933. For retrospective determination for subject-level endovascular treatment eligibility, the DL algorithm achieved an accuracy of 92%, with a sensitivity of 0.89 and specificity of 0.95. When applied to the GE pCASL data, the DL algorithm achieved a voxel-wise area under the curve of 0.94 and a subject-level accuracy of 92% for endovascular treatment eligibility. Conclusions- pCASL perfusion magnetic resonance imaging in conjunction with the DL algorithm provides a promising approach for assisting decision-making for endovascular treatment in patients with acute ischemic stroke.

Dataset Information

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms.

Publications

Effects of Study Population, Labeling and Training on Glaucoma Detection Using Deep Learning Algorithms.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets