Dataset Information

Evaluation of an AI system for the detection of diabetic retinopathy from images captured with a handheld portable fundus camera: the MAILOR AI study.

ABSTRACT:

Objectives

To evaluate the performance of an artificial intelligence (AI) system (Pegasus, Visulytix Ltd., UK*) at the detection of diabetic retinopathy (DR) from images captured by a handheld portable fundus camera.

Methods

A cohort of 6404 patients (~80% with diabetes mellitus) was screened for retinal diseases using a handheld portable fundus camera (Pictor Plus, Volk Optical Inc., USA) at the Mexican Advanced Imaging Laboratory for Ocular Research. The images were graded for DR by specialists according to the Scottish DR grading scheme. The performance of the AI system was evaluated, retrospectively, in assessing referable DR (RDR) and proliferative DR (PDR) and compared with the performance on a publicly available desktop camera benchmark dataset.

Results

For RDR detection, Pegasus performed with an 89.4% (95% CI: 88.0-90.7) area under the receiver operating characteristic (AUROC) curve for the MAILOR cohort, compared with an AUROC of 98.5% (95% CI: 97.8-99.2) on the benchmark dataset. This difference was statistically significant. Moreover, no statistically significant difference was found in performance for PDR detection with Pegasus achieving an AUROC of 94.3% (95% CI: 91.0-96.9) on the MAILOR cohort and 92.2% (95% CI: 89.4-94.8) on the benchmark dataset.

Conclusions

Pegasus showed good transferability for the detection of PDR from a curated desktop fundus camera dataset to real-world clinical practice with a handheld portable fundus camera. However, there was a substantial, and statistically significant, decrease in the diagnostic performance for RDR when using the handheld device.

SUBMITTER: Rogers TW

PROVIDER: S-EPMC8026959 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Topological characterization of the Retinal microvascular nEtwork visualized by portable fuNDus camera (TREND) is a database comprising of 72 color digital retinal images collected from the students of the Faculty of Medicine at the University of Montenegro, in the period from February 18th to March 11th 2020. The database also includes binarized images of manually segmented microvascular networks associated with each raw image. The participant demographic characteristics, health status, and social habits information such as age, sex, body mass index, smoking history, alcohol use, as well as previous medical history was collected. As proof of the concept, a smaller set of 10 color digital fundus images from healthy older participants is also included. Comparison of the microvascular parameters of these two sets of images demonstrate that digital fundus images recorded with a hand-held portable camera are able to capture the changes in patterns of microvascular network associated with aging. The raw images from the TREND database provide a standard that defines normal retinal anatomy and microvascular network geometry in young healthy people in Montenegro as it is seen with the digital hand-held portable non-mydriatic MiiS HORUS Scope DEC 200.This knowledge could facilitate the application of this technology at the primary level of health care for large scale telematic screening for complications of chronic diseases, such as hypertensive and diabetic retinopathy. In addition, it could aid in the development of new methods for early detection of age-related changes in the retina, systemic chronic diseases, as well as eye-specific diseases. The associated manually segmented images of the microvascular networks provide the standard that can be used for development of automatic software for image quality assessment, segmentation of microvascular network, and for computer-aided detection of pathological changes in retina. The TREND database is freely available at https://doi.org/10.5281/zenodo.4521043.

Project description:Retinal blood vessels are considered valuable biomarkers for the detection of diabetic retinopathy, hypertensive retinopathy, and other retinal disorders. Ophthalmologists analyze retinal vasculature by manual segmentation, which is a tedious task. Numerous studies have focused on automatic retinal vasculature segmentation using different methods for ophthalmic disease analysis. However, most of these methods are computationally expensive and lack robustness. This paper proposes two new shallow deep learning architectures: dual-stream fusion network (DSF-Net) and dual-stream aggregation network (DSA-Net) to accurately detect retinal vasculature. The proposed method uses semantic segmentation in raw color fundus images for the screening of diabetic and hypertensive retinopathies. The proposed method's performance is assessed using three publicly available fundus image datasets: Digital Retinal Images for Vessel Extraction (DRIVE), Structured Analysis of Retina (STARE), and Children Heart Health Study in England Database (CHASE-DB1). The experimental results revealed that the proposed method provided superior segmentation performance with accuracy (Acc), sensitivity (SE), specificity (SP), and area under the curve (AUC) of 96.93%, 82.68%, 98.30%, and 98.42% for DRIVE, 97.25%, 82.22%, 98.38%, and 98.15% for CHASE-DB1, and 97.00%, 86.07%, 98.00%, and 98.65% for STARE datasets, respectively. The experimental results also show that the proposed DSA-Net provides higher SE compared to the existing approaches. It means that the proposed method detected the minor vessels and provided the least false negatives, which is extremely important for diagnosis. The proposed method provides an automatic and accurate segmentation mask that can be used to highlight the vessel pixels. This detected vasculature can be utilized to compute the ratio between the vessel and the non-vessel pixels and distinguish between diabetic and hypertensive retinopathies, and morphology can be analyzed for related retinal disorders.