Dataset Information

Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.

ABSTRACT: To increase the efficiency of retinal image grading, algorithms for automated grading have been developed, such as the IDx-DR 2.0 device. We aimed to determine the ability of this device, incorporated in clinical work flow, to detect retinopathy in persons with type 2 diabetes.Retinal images of persons treated by the Hoorn Diabetes Care System (DCS) were graded by the IDx-DR device and independently by three retinal specialists using the International Clinical Diabetic Retinopathy severity scale (ICDR) and EURODIAB criteria. Agreement between specialists was calculated. Results of the IDx-DR device and experts were compared using sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), distinguishing between referable diabetic retinopathy (RDR) and vision-threatening retinopathy (VTDR). Area under the receiver operating characteristic curve (AUC) was calculated.Of the included 1415 persons, 898 (63.5%) had images of sufficient quality according to the experts and the IDx-DR device. Referable diabetic retinopathy (RDR) was diagnosed in 22 persons (2.4%) using EURODIAB and 73 persons (8.1%) using ICDR classification. Specific intergrader agreement ranged from 40% to 61%. Sensitivity, specificity, PPV and NPV of IDx-DR to detect RDR were 91% (95% CI: 0.69-0.98), 84% (95% CI: 0.81-0.86), 12% (95% CI: 0.08-0.18) and 100% (95% CI: 0.99-1.00; EURODIAB) and 68% (95% CI: 0.56-0.79), 86% (95% CI: 0.84-0.88), 30% (95% CI: 0.24-0.38) and 97% (95% CI: 0.95-0.98; ICDR). The AUC was 0.94 (95% CI: 0.88-1.00; EURODIAB) and 0.87 (95% CI: 0.83-0.92; ICDR). For detection of VTDR, sensitivity was lower and specificity was higher compared to RDR. AUC's were comparable.Automated grading using the IDx-DR device for RDR detection is a valid method and can be used in primary care, decreasing the demand on ophthalmologists.

SUBMITTER: van der Heijden AA

PROVIDER: S-EPMC5814834 | biostudies-other | 2018 Feb

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.

van der Heijden Amber A AA Abramoff Michael D MD Verbraak Frank F van Hecke Manon V MV Liem Albert A Nijpels Giel G

Acta ophthalmologica 20171127 1

<h4>Purpose</h4>To increase the efficiency of retinal image grading, algorithms for automated grading have been developed, such as the IDx-DR 2.0 device. We aimed to determine the ability of this device, incorporated in clinical work flow, to detect retinopathy in persons with type 2 diabetes.<h4>Methods</h4>Retinal images of persons treated by the Hoorn Diabetes Care System (DCS) were graded by the IDx-DR device and independently by three retinal specialists using the International Clinical Dia ...[more]

PMID: 29178249

Similar Datasets

Project description:ImportanceScreening for diabetic retinopathy is recommended for children with type 1 diabetes (T1D) and type 2 diabetes (T2D), yet screening rates remain low. Point-of-care diabetic retinopathy screening using autonomous artificial intelligence (AI) has become available, providing immediate results in the clinic setting, but the cost-effectiveness of this strategy compared with standard examination is unknown.ObjectiveTo assess the cost-effectiveness of detecting and treating diabetic retinopathy and its sequelae among children with T1D and T2D using AI diabetic retinopathy screening vs standard screening by an eye care professional (ECP).Design, setting, and participantsIn this economic evaluation, parameter estimates were obtained from the literature from 1994 to 2019 and assessed from March 2019 to January 2020. Parameters included out-of-pocket cost for autonomous AI screening, ophthalmology visits, and treating diabetic retinopathy; probability of undergoing standard retinal examination; relative odds of undergoing screening; and sensitivity, specificity, and diagnosability of the ECP screening examination and autonomous AI screening.Main outcomes and measuresCosts or savings to the patient based on mean patient payment for diabetic retinopathy screening examination and cost-effectiveness based on costs or savings associated with the number of true-positive results identified by diabetic retinopathy screening.ResultsIn this study, the expected true-positive proportions for standard ophthalmologic screening by an ECP were 0.006 for T1D and 0.01 for T2D, and the expected true-positive proportions for autonomous AI were 0.03 for T1D and 0.04 for T2D. The base case scenario of 20% adherence estimated that use of autonomous AI would result in a higher mean patient payment ($8.52 for T1D and $10.85 for T2D) than conventional ECP screening ($7.91 for T1D and $8.20 for T2D). However, autonomous AI screening was the preferred strategy when at least 23% of patients adhered to diabetic retinopathy screening.Conclusions and relevanceThese results suggest that point-of-care diabetic retinopathy screening using autonomous AI systems is effective and cost saving for children with diabetes and their caregivers at recommended adherence rates.

Project description:Purpose: To assess the accuracy and efficacy of a semi-automated deep learning algorithm (DLA) assisted approach to detect vision-threatening diabetic retinopathy (DR). Methods: We developed a two-step semi-automated DLA-assisted approach to grade fundus photographs for vision-threatening referable DR. Study images were obtained from the Lingtou Cohort Study, and captured at participant enrollment in 2009-2010 ("baseline images") and annual follow-up between 2011 and 2017. To begin, a validated DLA automatically graded baseline images for referable DR and classified them as positive, negative, or ungradable. Following, each positive image, all other available images from patients who had a positive image, and a 5% random sample of all negative images were selected and regraded by trained human graders. A reference standard diagnosis was assigned once all graders achieved consistent grading outcomes or with a senior ophthalmologist's final diagnosis. The semi-automated DLA assisted approach combined initial DLA screening and subsequent human grading for images identified as high-risk. This approach was further validated within the follow-up image datasets and its time and economic costs evaluated against fully human grading. Results: For evaluation of baseline images, a total of 33,115 images were included and automatically graded by the DLA. 2,604 images (480 positive results, 624 available other images from participants with a positive result, and 1500 random negative samples) were selected and regraded by graders. The DLA achieved an area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and accuracy of 0.953, 0.970, 0.879, and 88.6%, respectively. In further validation within the follow-up image datasets, a total of 88,363 images were graded using this semi-automated approach and human grading was performed on 8975 selected images. The DLA achieved an AUC, sensitivity, and specificity of 0.914, 0.852, 0.853, respectively. Compared against fully human grading, the semi-automated DLA-assisted approach achieved an estimated 75.6% time and 90.1% economic cost saving. Conclusions: The DLA described in this study was able to achieve high accuracy, sensitivity, and specificity in grading fundus images for referable DR. Validated against long-term follow-up datasets, a semi-automated DLA-assisted approach was able to accurately identify suspect cases, and minimize misdiagnosis whilst balancing safety, time, and economic cost.

Project description:BackgroundDespite recent incentives through Medicare (Australia's universal health insurance scheme) to increase retinal screening rates in primary care, comprehensive diabetic retinopathy (DR) screening has not been reached in Australia. The current study aimed to identify key factors affecting the delivery of diabetic retinopathy (DR) screening in Australian general practices.MethodsA descriptive qualitative study involving in-depth interviews was carried out from November 2019 to March 2020. Using purposive snowballing sampling, 15 general practitioners (GPs) were recruited from urban and rural general practices in New South Wales and Western Australia. A semi-structured interview guide was used to collect data from participants. All interviews were conducted over the phone by one facilitator, and each interview lasted up to 45 min. The Socio-Ecological Model was used to inform the content of the interview topic guides and subsequent data analysis. Recorded data were transcribed verbatim, and thematic analysis was conducted to identify and classify recurrent themes.ResultsOf 15 GPs interviewed, 13 were male doctors, and the mean age was 54.7 ± 15.5 years. Seven participants were practising in urban areas, while eight were practising in regional or remote areas. All participants had access to a direct ophthalmoscope, but none owned retinal cameras. None of the participants reported performing DR screening. Only three participants were aware of the Medicare Benefits Schedule (MBS) items 12,325 and 12,326 that allow GPs to bill for retinal screening. Seven themes, a combination of facilitators and barriers, emerged from interviews with the GPs. Despite the strong belief in their role in managing chronic diseases, barriers such as costs of retinal cameras, time constraints, lack of skills to make DR diagnosis, and unawareness of Medicare incentives for non-mydriatic retinal photography made it difficult to conduct DR screening in general practice. However, several enabling strategies to deliver DR screening within primary care include increasing GPs' access to continuing professional development, subsidising the cost of retinal cameras, and the need for a champion ace to take the responsibility of retinal photography.ConclusionThis study identified essential areas at the system level that require addressing to promote the broader implementation of DR screening, in particular, a nationwide awareness campaign to maximise the use of MBS items, improve GPs' competency, and subsidise costs of the retinal cameras for small and rural general practices.

Project description:BackgroundThe automated screening of patients at risk of developing diabetic retinopathy represents an opportunity to improve their midterm outcome and lower the public expenditure associated with direct and indirect costs of common sight-threatening complications of diabetes.ObjectiveThis study aimed to develop and evaluate the performance of an automated deep learning-based system to classify retinal fundus images as referable and nonreferable diabetic retinopathy cases, from international and Mexican patients. In particular, we aimed to evaluate the performance of the automated retina image analysis (ARIA) system under an independent scheme (ie, only ARIA screening) and 2 assistive schemes (ie, hybrid ARIA plus ophthalmologist screening), using a web-based platform for remote image analysis to determine and compare the sensibility and specificity of the 3 schemes.MethodsA randomized controlled experiment was performed where 17 ophthalmologists were asked to classify a series of retinal fundus images under 3 different conditions. The conditions were to (1) screen the fundus image by themselves (solo); (2) screen the fundus image after exposure to the retina image classification of the ARIA system (ARIA answer); and (3) screen the fundus image after exposure to the classification of the ARIA system, as well as its level of confidence and an attention map highlighting the most important areas of interest in the image according to the ARIA system (ARIA explanation). The ophthalmologists' classification in each condition and the result from the ARIA system were compared against a gold standard generated by consulting and aggregating the opinion of 3 retina specialists for each fundus image.ResultsThe ARIA system was able to classify referable vs nonreferable cases with an area under the receiver operating characteristic curve of 98%, a sensitivity of 95.1%, and a specificity of 91.5% for international patient cases. There was an area under the receiver operating characteristic curve of 98.3%, a sensitivity of 95.2%, and a specificity of 90% for Mexican patient cases. The ARIA system performance was more successful than the average performance of the 17 ophthalmologists enrolled in the study. Additionally, the results suggest that the ARIA system can be useful as an assistive tool, as sensitivity was significantly higher in the experimental condition where ophthalmologists were exposed to the ARIA system's answer prior to their own classification (93.3%), compared with the sensitivity of the condition where participants assessed the images independently (87.3%; P=.05).ConclusionsThese results demonstrate that both independent and assistive use cases of the ARIA system present, for Latin American countries such as Mexico, a substantial opportunity toward expanding the monitoring capacity for the early detection of diabetes-related blindness.

Dataset Information

Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.

Publications

Validation of automated screening for referable diabetic retinopathy with the IDx-DR device in the Hoorn Diabetes Care System.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets