The highly regarded Research Labs at Google are charged with “tackling the most challenging problems in computer science and related fields,” including eye care and ophthalmology. A groundbreaking project, announced in 2014 and still in development, was the creation of a prototype “smart” contact lens to monitor blood glucose levels continuously for people with diabetes.
Most recently, in this month’s Journal of the American Medical Association, Google Research Labs announced yet another groundbreaking diabetes-related advance: Using machine learning and artificial intelligence to screen for diabetic retinopathy and diabetic macular edema in patients with diabetes. According to Google, “Automated diabetic retinopathy screening methods with high accuracy have the strong potential to assist doctors in evaluating more patients and quickly routing those who need help to a specialist.”
Although this is a potentially significant breakthrough, study lead authors Lily Peng, M.D., Ph.D. and Varun Gulshan, Ph.D. urge caution, noting that “These are exciting results, but there is still a lot of work to do. Further research is necessary to determine the feasibility of applying this [screening technique] in the clinical setting and to determine whether [its use] could lead to improved care and outcomes compared with current ophthalmologic assessment.”
The Research from the Journal of the American Medical Association (JAMA)
This new Google research, titled Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs, has been published “online first” as a freely available article in the November 29, 2016 edition of JAMA, an international peer-reviewed journal published monthly by the American Medical Association and the most widely circulated medical journal in the world.
The authors are Varun Gulshan, PhD; Lily Peng, MD, PhD; Marc Coram, PhD; Martin C. Stumpe, PhD; Derek Wu, BS; Arunachalam Narayanaswamy, PhD; Subhashini Venugopala, MS; Kasumi Widner, MS; Tom Madams, MEng; Jorge Cuadros, OD, PhD; Ramasamy Kim, OD, DNB; Rajiv Raman, MS, DNB; Philip C. Nelson, BS; Jessica L. Mega, MD, MPH; and Dale R. Webster, PhD.
They represent the following organizations and institutions: Google Inc, Mountain View, CA; the University of Texas, Austin; EyePACS LLC, San Jose, CA; the University of California, Berkeley; the Aravind Medical Research Foundation, Madurai, India; Shri Bhagwan Mahavir Vitreoretinal Services, Chennai, India; Verily Life Sciences, Mountain View, CA; and Brigham and Women’s Hospital and Harvard Medical School, Boston, MA.
First, Some Terminology
Here is a brief explanation of some key terms used in this artificial intelligence research:
- Machine learning: A type of artificial intelligence that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that can teach themselves to grow and change when exposed to new data. (via WhatIs.com)
- Algorithm: A step-by-step problem-solving procedure.
- Fundus: The interior rear surface of the eye, which includes the retina, optic disc, and macula.
- Fundus camera: A specialized low-power microscope with an attached camera designed to photograph the fundus; i.e., a retinal fundus photograph.
About the Research
Excerpted from Deep Learning for Detection of Diabetic Eye Disease, via the Google Research Blog:
Diabetic retinopathy (DR) is the fastest growing cause of blindness, with nearly 415 million diabetic patients at risk worldwide. If caught early, the disease can be treated; if not, it can lead to irreversible blindness. Unfortunately, medical specialists capable of detecting the disease are not available in many parts of the world where diabetes is prevalent. We believe that Machine Learning can help doctors identify patients in need, particularly among underserved populations.
A few years ago, several of us began wondering if there was a way Google technologies could improve the DR screening process, specifically by taking advantage of recent advances in Machine Learning and Computer Vision. In [our article], we present a deep learning algorithm capable of interpreting signs of DR in retinal photographs, potentially helping doctors screen more patients in settings with limited resources.
One of the most common ways to detect diabetic eye disease is to have a specialist examine pictures of the back of the eye and rate them for disease presence and severity…. Interpreting these photographs requires specialized training, and in many regions of the world there aren’t enough qualified graders to screen everyone who is at risk.
Working closely with doctors both in India and the US, we created a development dataset of 128,000 images which were each evaluated by 3-7 ophthalmologists from a panel of 54 ophthalmologists. This dataset was used to train a deep neural network to detect referable diabetic retinopathy.
We then tested the algorithm’s performance on two separate clinical validation sets totaling [approximately] 12,000 images, with the majority decision of a panel 7 or 8 U.S. board-certified ophthalmologists serving as the reference standard. The ophthalmologists selected for the validation sets were the ones that showed high consistency from the original group of 54 doctors. The results [reported in more depth below] show that our algorithm’s performance is on-par with that of ophthalmologists.
These are exciting results, but there is still a lot of work to do. First, while the conventional quality measures we used to assess our algorithm are encouraging, we are working with retinal specialists to define even more robust reference standards that can be used to quantify performance.
Furthermore, interpretation of a 2-D fundus photograph, which we demonstrate in this paper, is only one part in a multi-step process that leads to a diagnosis for diabetic eye disease. In some cases, doctors use a 3-D imaging technology [i.e, optical coherence tomography] to examine various layers of a retina in detail.
More about Diabetic Eye Disease
Diabetic Retinopathy
Although people with diabetes are more likely to develop cataracts at a younger age and are twice as likely to develop glaucoma as people who do not have diabetes, the primary vision problem caused by diabetes is diabetic retinopathy, the leading cause of new cases of blindness and low vision in adults aged 20-65:
diabetic retinopathy might see
- “Retinopathy” is a general term that describes damage to the retina.
- The retina is a thin, light-sensitive tissue that lines the inside surface of the eye. Nerve cells in the retina convert incoming light into electrical impulses. These electrical impulses are carried by the optic nerve to the brain, which interprets them as visual images.
- Diabetic retinopathy occurs when there is damage to the small blood vessels that nourish tissue and nerve cells in the retina.
- “Proliferative” is a general term that means to grow or increase at a rapid rate by producing new tissue or cells. When the term “proliferative” is used in relation to diabetic retinopathy, it describes the growth, or proliferation, of abnormal new blood vessels in the retina. “Non-proliferative” indicates that this process is not yet occurring.
- Proliferative diabetic retinopathy affects approximately 1 in 20 individuals with the disease.
Four Stages of Diabetic Retinopathy
According to the National Eye Institute, diabetic retinopathy has four stages:
- Mild non-proliferative retinopathy: At this early stage, small areas of balloon-like swelling occur in the retina’s tiny blood vessels.
- Moderate non-proliferative retinopathy: As the disease progresses, some blood vessels that nourish the retina become blocked.
- Severe non-proliferative retinopathy: Many more blood vessels become blocked, which disrupts the blood supply that nourishes the retina. The damaged retina then signals the body to produce new blood vessels.
- Proliferative retinopathy: At this advanced stage, signals sent by the retina trigger the development of new blood vessels that grow (or proliferate) in the retina and the vitreous, which is a transparent gel that fills the interior of the eye. Because these new blood vessels are abnormal, they can rupture and bleed, causing hemorrhages in the retina or vitreous. Scar tissue can develop and can tug at the retina, causing further damage or even retinal detachment.
Diagnosing Diabetic Eye Disease
Diabetic retinopathy usually has no early warning signs. It can be detected only through a comprehensive eye examination that looks for early signs of the disease, including:
- Leaking blood vessels
- Macular edema (swelling)
- Pale, fatty deposits on the retina
- Damaged nerve tissue
- Any changes to the retinal blood vessels
To diagnose diabetic eye disease effectively, eye care specialists recommend a comprehensive diabetic eye examination that includes the following procedures:
- Distance and near vision acuity tests
- A dilated eye (or fundus) examination, which includes the use of an ophthalmoscope. In a dilated eye examination, it is the pupil that is dilated—not the entire eye. This allows the examiner to see through the pupil to the retina. Visual acuity tests alone are not sufficient to detect diabetic retinopathy in its early stages.
- A tonometry test to measure fluid pressure inside the eye.
- A fluorescein angiography test, if more serious retinal changes, such as macular edema, are suspected. Fluorescein angiography is an eye test that uses a special dye and camera to look at blood flow in the retina.
- Optical coherence tomography (OCT) testing may be used to gain a clearer picture of the retina and its supporting layers. OCT is a type of medical imaging technology that produces high-resolution cross-sectional and three-dimensional images of the eye.
More about the Study from JAMA
Excerpted from the study’s Key Points and Abstract, with the full article, including outcome measures and results, available online:
Question: How does the performance of an automated deep learning algorithm compare with manual grading by ophthalmologists for identifying diabetic retinopathy in retinal fundus photographs?
Objective: To apply deep learning to create an algorithm for automated detection of diabetic retinopathy and diabetic macular edema in retinal fundus photographs.
Design and Setting: A specific type of neural network optimized for image classification called a deep convolutional neural network was trained using a retrospective [i.e., past records] development data set of 128,175 retinal images, which were graded 3 to 7 times for diabetic retinopathy, diabetic macular edema, and image gradability by a panel of 54 US licensed ophthalmologists and ophthalmology senior residents between May and December 2015. The resultant algorithm was validated in January and February 2016 using 2 separate data sets, both graded by at least 7 US board-certified ophthalmologists with high intragrader consistency.
Main Outcomes and Measures: The sensitivity and specificity of the algorithm for detecting referable diabetic retinopathy (RDR), defined as moderate and worse diabetic retinopathy, referable diabetic macular edema, or both, were generated based on the reference standard of the majority decision of the ophthalmologist panel. The algorithm was evaluated at 2 operating points selected from the development set, one selected for high specificity and another for high sensitivity.
Finding: In 2 validation sets of 9,963 images and 1,748 images, at the operating point selected for high specificity, the algorithm had 90.3% and 87.0% sensitivity and 98.1% and 98.5% specificity for detecting referable diabetic retinopathy, defined as moderate or worse diabetic retinopathy or referable macular edema by the majority decision of a panel of at least 7 US board-certified ophthalmologists. At the operating point selected for high sensitivity, the algorithm had 97.5% and 96.1% sensitivity and 93.4% and 93.9% specificity in the 2 validation sets.
Meaning: Deep learning algorithms had high sensitivity and specificity for detecting diabetic retinopathy and macular edema in retinal fundus photographs.