Dataset Information

THEIA™ development, and testing of artificial intelligence-based primary triage of diabetic retinopathy screening images in New Zealand.

ABSTRACT:

Aim

To develop and evaluate an artificial intelligence triage system with high sensitivity for detecting referable diabetic retinopathy and maculopathy, while maintaining high specificity for non-referable disease, for clinical implementation within the New Zealand national diabetic retinopathy screening programme.

Methods

The THEIA™ artificial intelligence system for retinopathy and maculopathy screening, was developed at Toku Eyes using routinely collected retinal screening datasets from two of the largest district health boards in Auckland, New Zealand: the Auckland District Health Board and the Counties Manukau District Health Board. All retinal images from consecutive individuals receiving retinal screening between January 2009 and December 2018 were used. Images were labelled as non-sight-threatening, potentially referable or sight-threatening for New Zealand implementation, or as referable (potentially referable + sight-threatening)/non-referable (non-sight-threatening) for global comparison.

Results

Data from 32 354 unique people with diabetes (63 843 when including multiple visits) were available, of which 95-97%, 0.9-2.4% and 1.1-3.1% were categorized as non-sight-threatening, potentially referable and sight-threatening, respectively. Using the referable/non-referable categories, THEIA achieved overall sensitivity of 94% (95% CI 92-95) in the Auckland District Health Board and 95% (95% CI 92-97) in the Counties Manukau District Health Board datasets, while preserving specificity of 63% (95% CI 62-64) for the Auckland District Health Board and 61% (95% CI 60-62) for the Counties Manukau District Health Board. Implementing THEIA into a New Zealand national diabetic screening programme could significantly reduce the manual grading load.

Conclusion

THEIA, an artificial intelligence tool to assist in clinical decision-making, tailored to the needs of the New Zealand national diabetic screening programme, delivered high sensitivity for detecting referable retinopathy within the multi-ethnic New Zealand population with diabetes.

SUBMITTER: Vaghefi E

PROVIDER: S-EPMC8048953 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionDiabetic retinopathy (DR) is the leading cause of preventable blindness in Saudi Arabia. With a prevalence of up to 40% of patients with diabetes, DR constitutes a significant public health burden on the country. Saudi Arabia has not yet established a national screening program for DR. Mounting evidence shows that Artificial intelligence (AI)-based DR screening programs are slowly becoming superior to traditional screening, with the COVID-19 pandemic accelerating research into this topic as well as changing the outlook of the public toward it. The main objective of this study is to evaluate the perception and acceptance of AI in DR screening among eye care professionals in Saudi Arabia.MethodsA cross-sectional study using a self-administered online-based questionnaire was distributed by email through the registry of the Saudi Commission For Health Specialties (SCFHS). 309 ophthalmologists and physicians involved in diabetic eye care in Saudi Arabia participated in the study. Data analysis was done by SPSS, and a value of p < 0.05 was considered significant for statistical purposes.Results54% of participants rated their level of AI knowledge as above average and 63% believed that AI and telemedicine are interchangeable. 66% believed that AI would decrease the workforce of physicians. 79% expected clinical efficiency to increase with AI. Around 50% of participants expected AI to be implemented in the next 5 years.DiscussionMost participants reported good knowledge about AI. Physicians with more clinical experience and those who used e-health apps in clinical practice regarded their AI knowledge as higher than their peers. Perceived knowledge was strongly related to acceptance of the benefits of AI-based DR screening. In general, there was a positive attitude toward AI-based DR screening. However, concerns related to the labor market and data confidentiality were evident. There should be further education and awareness about the topic.

Project description:BackgroundDiabetic retinopathy (DR) is a leading cause of blindness. Our objective was to evaluate the performance of an artificial intelligence (AI) system integrated into a handheld smartphone-based retinal camera for DR screening using a single retinal image per eye.MethodsImages were obtained from individuals with diabetes during a mass screening program for DR in Blumenau, Southern Brazil, conducted by trained operators. Automatic analysis was conducted using an AI system (EyerMaps™, Phelcom Technologies LLC, Boston, USA) with one macula-centered, 45-degree field of view retinal image per eye. The results were compared to the assessment by a retinal specialist, considered as the ground truth, using two images per eye. Patients with ungradable images were excluded from the analysis.ResultsA total of 686 individuals (average age 59.2 ± 13.3 years, 56.7% women, diabetes duration 12.1 ± 9.4 years) were included in the analysis. The rates of insulin use, daily glycemic monitoring, and systemic hypertension treatment were 68.4%, 70.2%, and 70.2%, respectively. Although 97.3% of patients were aware of the risk of blindness associated with diabetes, more than half of them underwent their first retinal examination during the event. The majority (82.5%) relied exclusively on the public health system. Approximately 43.4% of individuals were either illiterate or had not completed elementary school. DR classification based on the ground truth was as follows: absent or nonproliferative mild DR 86.9%, more than mild (mtm) DR 13.1%. The AI system achieved sensitivity, specificity, positive predictive value, and negative predictive value percentages (95% CI) for mtmDR as follows: 93.6% (87.8-97.2), 71.7% (67.8-75.4), 42.7% (39.3-46.2), and 98.0% (96.2-98.9), respectively. The area under the ROC curve was 86.4%.ConclusionThe portable retinal camera combined with AI demonstrated high sensitivity for DR screening using only one image per eye, offering a simpler protocol compared to the traditional approach of two images per eye. Simplifying the DR screening process could enhance adherence rates and overall program coverage.

Project description:IntroductionGlobally, diabetic retinopathy (DR) is a major cause of blindness. Sub-Saharan Africa is projected to see the largest proportionate increase in the number of people living with diabetes over the next two decades. Screening for DR is recommended to prevent sight loss; however, in many low and middle-income countries, because of a lack of specialist eye care staff, current screening services for DR are not optimal. The use of artificial intelligence (AI) for DR screening, which automates the grading of retinal photographs and provides a point-of-screening result, offers an innovative potential solution to improve DR screening in Tanzania.Methods and analysisWe will test the hypothesis that AI-supported DR screening increases the proportion of persons with true referable DR who attend the central ophthalmology clinic following referral after screening in a single-masked, parallel group, individually randomised controlled trial. Participants (2364) will be randomised (1:1 ratio) to either AI-supported or the standard of care DR screening pathway. Participants allocated to the AI-supported screening pathway will receive their result followed by point-of-screening counselling immediately after retinal image capture. Participants in the standard of care arm will receive their result and counselling by phone once the retinal images have been graded in the usual way (typically after 2-4 weeks). The primary outcome is the proportion of persons with true referable DR attending the central ophthalmology clinic within 8 weeks of screening. Secondary outcomes, by trial arm, include the proportion of persons attending the central ophthalmology clinic out of all those referred, sensitivity and specificity, number of false positive referrals, acceptability and fidelity of AI-supported screening.Ethics and disseminationThe London School of Hygiene & Tropical Medicine, Kilimanjaro Christian Medical Centre and Tanzanian National Institute of Medical Research ethics committees have approved the trial. The results will be submitted to peer-reviewed journals for publication.Trial registration numberISRCTN18317152.

Project description:This study investigated the diagnostic performance, feasibility, and end-user experiences of an artificial intelligence (AI)-assisted diabetic retinopathy (DR) screening model in real-world Australian healthcare settings. The study consisted of two components: (1) DR screening of patients using an AI-assisted system and (2) in-depth interviews with health professionals involved in implementing screening. Participants with type 1 or type 2 diabetes mellitus attending two endocrinology outpatient and three Aboriginal Medical Services clinics between March 2018 and May 2019 were invited to a prospective observational study. A single 45-degree (macula centred), non-stereoscopic, colour retinal image was taken of each eye from participants and were instantly screened for referable DR using a custom offline automated AI system. A total of 236 participants, including 174 from endocrinology and 62 from Aboriginal Medical Services clinics, provided informed consent and 203 (86.0%) were included in the analysis. A total of 33 consenting participants (14%) were excluded from the primary analysis due to ungradable or missing images from small pupils (n = 21, 63.6%), cataract (n = 7, 21.2%), poor fixation (n = 2, 6.1%), technical issues (n = 2, 6.1%), and corneal scarring (n = 1, 3%). The area under the curve, sensitivity, and specificity of the AI system for referable DR were 0.92, 96.9% and 87.7%, respectively. There were 51 disagreements between the reference standard and index test diagnoses, including 29 which were manually graded as ungradable, 21 false positives, and one false negative. A total of 28 participants (11.9%) were referred for follow-up based on new ocular findings, among whom, 15 (53.6%) were able to be contacted and 9 (60%) adhered to referral. Of 207 participants who completed a satisfaction questionnaire, 93.7% specified they were either satisfied or extremely satisfied, and 93.2% specified they would be likely or extremely likely to use this service again. Clinical staff involved in screening most frequently noted that the AI system was easy to use, and the real-time diagnostic report was useful. Our study indicates that AI-assisted DR screening model is accurate and well-accepted by patients and clinicians in endocrinology and indigenous healthcare settings. Future deployments of AI-assisted screening models would require consideration of downstream referral pathways.

Project description:ObjectiveDiabetic retinopathy (DR) may lead to irreversible damage to the eye and cause blindness if diagnosed in its advanced stages. Artificial intelligence (AI) may support screening and contribute to a timely diagnosis. The aim of this study was to evaluate factors that might influence the success of implementing AI-supported devices for DR screenings in general practice.MethodsA questionnaire with modules on attitudes toward digital solutions, technical factors, perceived patient perspectives, and sociodemographic data was constructed and 2100 general practitioners (GPs) in Germany were invited to participate via a personal letter.ResultsTwo hundred nine physicians participated in the survey (10% response rate, mean age = 54 years, 46% women). Acquisition costs (mean = 1.37), remuneration (mean = 1.46), and running costs (mean = 1.40) were considered particularly relevant in the context of AI-based screening tools. GPs indicated that a mean of €27.00 (SD = 19) was considered to be an appropriate reimbursement for an AI-based screening for DR in their practice. Less relevant factors were availability of a smartphone used in the practice (mean = 2.53) and time until the examination result was available (mean = 2.29). Important technical factors were practicability of the device (mean = 1.27), unproblematic installation of any necessary software (mean = 1.34), and the integrability into the practice information system (mean = 1.44). Considering the patient welfare, physicians rated the accuracy of the examination, omission of pupil dilation, and the duration of the examination as the most important factors. Participants ranked the factors broadening the scope of care, strengthening the primary care (PC) range, and signs of modern medical practice as the most important factors for making an AI-based screening tool attractive for their practice.ConclusionsThese findings serve as a basis for a successful implementation of AI-assisted screening devices in PC and might facilitate early screenings for ophthalmological diseases in general practice. The most relevant barriers that need to be overcome for a successful implementation of such tools include clarification of the costs and reimbursement policies.