Using artificial intelligence to automate screening for diabetic retinopathy

A Deep Learning System (DLS) is an advanced form of artificial intelligence (AI) that can start with a mix of images from different groups and then learn to recognize unique features that organize each image into their own group. DLS functions have been widely adopted for use in multiple technologies, including face and voice recognition, and are also being applied in medicine.

The ability of a DLS to recognize disease features in a medical image could aid physicians with clinical diagnoses, and the number of studies exploring medical AI systems has exploded over the last few years, with less than 50 in 2015 to more than 200 just a year later in 2016. Medical AI is already being developed or used in fields like pathology or neurology, but there is also a need for AI-based diagnostic systems in other fields like ophthalmology, where potential is especially great because images are acquired through fairly standardized methods across clinics.

A deep learning system to detect diabetic retinopathy

Diabetic eye diseases are a leading cause of blindness around the world, and research teams are developing DLS that could enhance the ease, efficiency, and accuracy of diagnosing diabetic retinopathy, including at different stages and with diabetic macular edema.

The Google Brain AI research team created a DLS that detects diabetic retinopathy and diabetic macular edema from fundus photographs, and which was also incorporated into clinics associated with the Aravind Eye Care System in India as part of an initiative to increase global access to eye care for patients with diabetes.^1,2

The AI company IDx also created a DLS for the detection of diabetic retinopathy, which is used in a primary care setting and provides results within a minute of submitting a fundus photo. Shortly after announcing positive results from a pivotal multicenter center, the FDA authorized marketing of the IDx diabetic retinopathy system for the autonomous detection of “more than mild” diabetic retinopathy in adults with diabetes.³

Another DLS was recently created with the goal of detecting multiple eye diseases, including two categories of diabetic retinopathy: referable and vision threatening. Referable diabetic retinopathy was defined as moderate non-proliferative diabetic retinopathy (NPDR) or worse, and could include diabetic macular edema. Vision-threatening diabetic retinopathy was defined as severe NPDR or proliferative diabetic retinopathy.⁴

A validation study was performed to evaluate the ability of the DLS to accurately detect referable and vision-threatening diabetic retinopathy from retinal fundus photographs. The Deep Eye Study was conducted at multiple international sites (Australia, China including Hong Kong, Mexico, Singapore, and the United States) and fundus photographs were acquired from multiple types of camera (Topcon, Canon, Zeiss, FundusVue). By including sites across the globe and acquiring images from multiple types of camera, the study was designed to produce results that would be generalizable to most clinics.⁴

The first step was for the DLS to learn how to differentiate among fundus images from people with referable, vision threatening, or no diabetic retinopathy, and 76,370 images were used in this initial training phase of the study. 112,648 fundus images were then used to test the system. The images came from the patient database of the Singapore National Diabetic Retinopathy Screening Program (SIDRP).⁴

In the testing phase, the accuracy of the DLS to detect diabetic retinopathy was compared to two senior professional graders with more than five years of experience and who were currently employed at the SIDRP for diabetic retinopathy screening. Accuracy was evaluated based on two measures: sensitivity and specificity. Sensitivity measures how many positive cases the system detects (and, importantly, how many positive cases the system misses), while specificity measures how well a system avoids false positives.⁴

For referable diabetic retinopathy, the study showed that the DLS was 90.5% sensitive and the graders were 91.1% sensitive, and that the difference was not statistically significant (p=.68). This means that the DLS and graders were equally good at catching referable diabetic retinopathy.⁴

The results also showed that the DLS was 91.6% specific for referable diabetic retinopathy, while the graders were 99.3% specific, and the difference was statistically significant in favor of the graders (p<.001). So this means that while the DLS was pretty good at avoiding false positive cases of referable diabetic retinopathy, the graders were nearly flawless.4

For vision-threatening diabetic retinopathy, the DLS was 100% sensitive and the graders were 88.5% sensitive, with a statistically significant difference in favor of the DLS (p<.001). The DLS nailed it and caught all cases of vision-threatening diabetic retinopathy in the test.⁴

The DLS was 91.1% specific in avoiding false positive cases of vision-threatening diabetic retinopathy but the graders were 99.6% specific, and the difference was statistically significant (p<.001). Just as with referable diabetic retinopathy, and even though the DLS had a high degree of specificity, the graders were better at keeping it limited to what they were supposed to be detecting.⁴

Automated screening for diabetic retinopathy in practice

Once you have a validated system, and data showing the relative sensitivities and specificities of the system versus manual graders, how do you incorporate a DLS like this into the clinic? A great example comes from the SIDRP who, based on the results of the Deep Eye Study, established the following diabetic-retinopathy screening algorithm for use in their own practice (Figure).⁵

1. Primary care clinic acquires retinal fundus image and sends to the SIDRP
2. First screening – by DLS
a. If DLS finds no mild diabetic retinopathy, schedule patient to be re-screen in one year
b. If DLS is positive for moderate NPDR or worse, advance to manual screening for confirmation
3. Second screening – by senior grader
a. If manual screening confirms a case of referable diabetic retinopathy, refer the patient to an ophthalmologist
b. If manual screening determines that the case is not referable, but rather no or mild diabetic retinopathy, then schedule patient to be re-screen in one year

SUMMARY

Diabetic eye diseases are a leading cause of blindness, and research teams are using deep learning AI to enhance the efficiency and accuracy of diagnosis. A recently validated DLS has been incorporated into the diabetic-retinopathy screening program at the SIDRP, and is an example of how DLS can be incorporated into clinical practice as ophthalmology AI tools become increasingly available.

Disclosures:

Rohit Varma, MD, MPH is the founding director of the USC Roski Eye Institute and a glaucoma specialist. He also holds the Grace and Emery Beardsley chair in Ophthalmology, and is a professor of Preventive Medicine at the Keck School of Medicine of University of Southern California in Los Angeles, California. Dr. Varma can be reached via e-mail at amravr@yahoo.com

References:

1. Gulshan V, Peng, L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402-2410.
2. Simonite T. Google’s AI eye doctor gets ready to go to work in India. Wired Magazine. June 8, 2017. www.wired.com/2017/06/googles-ai-eye-doctor-gets-ready-go-work-india/. Accessed July 23, 2018.
3. FDA permits marketing of IDx-DR for automated detection of diabetic retinopathy in primary care [press release]. IDx. April 12, 2018. www.eyediagnosis.net/single-post/2018/04/12/FDA-permits-marketing-of-IDx-DR-for-automated-detection-of-diabetic-retinopathy-in-primary-care. Accessed July 23, 2018.
4. Ting DSW, Cheung C Y-L, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318:2211-2223.
5. Ting DSW. Telemedicine and artificial intelligence using deep learning systems for tele-retinal diabetic retinopathy screening program. Paper presented at: The Association for Research in Vision and Ophthalmology (ARVO) 2018 Annual Meeting. April 29, 2018; Honolulu, HI.