Diabetic retinopathy AI: what held outside the trial, and who came back

In 2018 the United States (US) Food and Drug Administration (FDA) authorised IDx-DR to read a retinal photograph for diabetic retinopathy with no clinician looking at the image. In its pivotal trial of 900 patients the system reached a sensitivity of 87.2 percent and a specificity of 90.7 percent ^[1]. Two Polish cohorts later ran the same product, now sold as LumineticsCore, on 807 and 758 patients. Sensitivity was 99 percent and 99.3 percent. Specificity was 68 percent and 69.0 percent ^{[2, 3]}.

Both sets of numbers come from the same software making the same decision, in different settings. Diabetic retinopathy is the one indication where the FDA allows an artificial intelligence (AI) decision without a clinician reading the image. Four systems are cleared to make it ^{[4, 5, 6]}. It has more deployment evidence than any other use of AI in eye care, so it is the best place to see what changes between a pivotal trial and an ordinary clinic.

The first issue of Second Look set out the thread of this series: accuracy in a validation study is not a better outcome in primary care. This issue follows a patient through an AI screening visit and looks at each step from the side of the optometrist or ophthalmologist who receives the referral.

What the software does in a diabetes clinic

The four cleared systems are LumineticsCore from Digital Diagnostics, formerly IDx-DR, EyeArt from Eyenuk, AEYE-DS from AEYE Health, and iPredict-DR from iHealthscreen, which was cleared on 2 July 2026 ^{[4, 5, 6]}. Each is software that reads the photographs from a fundus camera. In the published deployments the camera sat in a general practice, diabetes or endocrinology clinic: a Topcon NW400 in the Polish diabetes clinics and a Topcon NW500 in an Israeli endocrinology clinic ^{[2, 7]}.

A visit runs like this. A member of the clinic staff photographs the retina, usually without dilating drops. In the Polish clinics that was two photographs of each eye, one centred on the optic disc and one on the macula ^[2]. The operator does not need to be an eye care professional. In the Israeli clinic the photographer was a novice with no ophthalmic training. In the Pilbara a research officer with no formal healthcare qualification took the photographs from a van ^{[7, 8]}.

The software reads the photographs during the visit and returns a decision rather than a grade. The threshold is more than mild diabetic retinopathy, which means moderate non-proliferative retinopathy or worse, or diabetic macular oedema, in either eye. Above it the patient is referred to an eye care professional. Below it no referral is made and the patient stays with the clinic. If the images cannot be read, the patient is dilated and photographed again ^{[1, 9]}. Some programmes refer an ungradable patient to eye care instead ^[10]. No clinician reads the image at any point.

This means two things for the optometrist. A negative result is not a normal retina, because mild non-proliferative retinopathy returns a negative by design. And the referral that reaches an eye care practice carries a finding made by software in another building, with no human grade attached.

England has looked at a different use. Eight systems carrying the European conformity (CE) mark were tested on the stored images of the North East London Diabetic Eye Screening Programme, against the programme's own human grading. The question was whether software could take the first read, so that human graders only look at the images it flags ^[11]. That is triage. People would still make the decision, and the test was run on past images, not in a live service. The evidence from both uses lines up against three steps of the visit.

• Can the photograph be read? The same product graded 64.9 percent of first images without dilation and 95.2 percent after a dilation protocol ^[9].
• Is the reading right? Sensitivity for moderate to severe disease held across eight systems in London, at 96.7 to 99.8 percent. Specificity for one product was 68 and 69.0 percent in Poland, against 90.7 percent in its pivotal trial ^{[11, 1, 2, 3]}.
• Does the patient come? After a positive screen, between 44.7 and 70.2 percent of patients attended, across four series that count different things ^{[7, 12, 13, 14]}.

None of these is a vision outcome. They are the steps a screening visit has to get through before a vision outcome is possible.

Step one: can the photograph be read?

The first decision the software makes is whether it can read the images at all. The IDx-DR version 2.3 clearance summary gives two gradability figures for the same 850 participants. The first is 64.9 percent on the first images without dilation. The second is 95.2 percent after a protocol that dilates when the first images fail ^[9]. Five US academic health systems that adopted LumineticsCore or EyeArt reported average non-mydriatic gradability of 49 to 75 percent ^[4].

Other clinics did better without drops. In the Israeli endocrinology clinic AEYE-DS returned a definitive result for 95.7 percent of 256 patients, with no dilation and a novice photographer ^[7]. In the Australian five-site study the AI found 13.4 percent of images ungradable and human graders found 15.6 percent ^[10]. In the Pilbara 8.4 percent of photographs were ungradable ^[8].

In a clinic that does not dilate, somewhere between one patient in twenty and one in two leaves without a result. That patient needs a second attempt, drops, or a referral for the photograph alone. The distance between 95 percent and 49 percent reflects the dilation protocol, the camera, the patients and operator training. It is not evidence that the software got worse.

Step two: is the reading right?

Where systems have been compared on the same images, sensitivity for moderate to severe disease held. Sensitivity for referable disease as a whole moved with the vendor. The London evaluation ran its eight systems over the images from 202,886 screening encounters ^[11].

• Sensitivity for moderate to severe retinopathy stayed at 96.7 to 99.8 percent.
• Sensitivity for any referable retinopathy ran from 83.7 to 98.7 percent between vendors.
• The false-positive rate ranged from 4.3 to 61.4 percent. That is the share of eyes with no retinopathy that a system flagged.

The Australian five-site study was labelled a pragmatic trial by its authors and was single-arm in design. It ran an AI-integrated non-mydriatic camera in general practice and endocrinology clinics. It reported a sensitivity of 83.7 percent (78.2 to 88.3) and a specificity of 96.1 percent (94.7 to 97.2) in 863 participants ^[10].

Specificity was lower in the two cohorts that tested IDx-DR outside its pivotal trial, and sensitivity was higher. The product reached a sensitivity of 87.2 percent and a specificity of 90.7 percent (88.3 to 92.7) in the 900-patient pivotal trial behind its 2018 FDA authorisation. When version 2.3 was run on the pivotal data for its 2022 clearance, with 850 participants evaluable, specificity was 90.07 percent (87.42 to 92.32) ^{[1, 9]}.

The first Polish cohort put it at a sensitivity of 99 percent and a specificity of 68 percent (64 to 72) in 807 patients. The second put it at 99.3 percent and 69.0 percent (65.2 to 72.6) in 758 patients. Both used consensus grading of the same photographs ^{[2, 3]}.

In Quebec, a different semi-automated system reached a specificity of 66.2 percent (54.3 to 76.3) in 115 patients against an ophthalmological examination ^[15]. However, the Polish cohorts found a similar specificity against a photographic standard, so a stricter reference standard does not fully explain it.

For the optometrist, specificity is the share of patients without the disease who are not sent on. At 68 percent, about three in ten disease-free patients who screened in those Polish clinics would have been referred to eye care. In London the false-positive rate ran from 4.3 to 61.4 percent between vendors, so the choice of vendor would largely set how many disease-free patients are referred.

Each positive screen means a confirmatory examination, and some of those patients do need treatment. In the Israeli clinic, four of the 34 screen-positive patients who were examined, 11.8 percent, needed anti-vascular endothelial growth factor injections or panretinal laser. The examination also picked up other eye conditions nobody had recorded ^[7].

Step three: does the patient come?

Getting the eye examination done is where the systems clearly do better than the old pathway. The ACCESS trial (AI for Children's diabetiC Eye ExamS) randomised young people with diabetes to one of two arms. One had an AI eye examination in their own diabetes clinic. The other had a scripted referral to eye care. Every one of the 81 in the AI arm completed the examination, against 22 percent of the 83 referred ^[12]. In specialist eye clinics in Bangladesh, clinic days randomised to an FDA-authorised autonomous system completed 1.59 patient encounters per specialist hour, against 1.14 without it ^[16].

Attendance after a positive result is less impressive. Across the four series in this article it ran from 44.7 to 70.2 percent, and the four count different things.

• In the Israeli clinic, 34 of 76 screen-positive patients, or 44.7 percent, were examined in the hospital's own retina service. Examinations elsewhere were not captured, so that figure is a floor ^[7].
• In ACCESS, 16 of the 25 young people with an abnormal result followed through, against 22 percent in the control arm ^[12].
• In the Stanford programme, 69.2 percent of 279 screen-positive patients saw an ophthalmologist within 90 days. Of those who went and answered a survey, 70.5 percent said the clinic's referral decided where ^[13].
• In a chart review of 225 patients in New Orleans, 29.8 percent of screen-positive patients did not attend the scheduled ophthalmology visit ^[14].

The finding that replicates is higher follow-up in pathways that include AI. Two pooled estimates give the same number on different scales, and they disagree on the reason. One meta-analysis of six studies and 20,108 patients gave a pooled odds ratio for follow-up of 1.89 (95 percent confidence interval 1.78 to 2.01). It attributed the gain to instant results ^[17].

A second, from Lions Outback Vision, pooled six studies to February 2025. It gave a relative risk of 1.89 with a far wider interval (1.18 to 3.03) and heterogeneity of 92 percent. An odds ratio and a relative risk of the same value do not describe the same size of effect. It credited the redesign of the care pathway around the AI, and not the AI itself ^[18].

For an eye care practice, the referral now comes from the diabetes clinic and the finding behind it is the software's. In the Stanford programme the clinic's referral also decided where most of those surveyed went. The patient who screened negative does not arrive at all. And a third to a half of the patients the software found did not reach an examination in the series above. That gap existed before the software.

In my own practice the same step fails. The patient leaves with the referral letter, and I find out at the next visit whether it was used.

What is on the Australian register

The Therapeutic Goods Administration (TGA) list of AI-enabled devices was last updated on 10 September 2026. It carries four entries that read a fundus photograph for diabetic retinopathy ^[19].

• TeleMedC's DR Grader screens and grades diabetic retinopathy from colour fundus photographs, in people with diabetes and in people not yet diagnosed.
• The iCare RETCAD software reads colour fundus photographs for age-related macular degeneration, diabetic retinopathy and glaucoma, as decision support.
• Two Eyetelligence products flag referable age-related macular degeneration, diabetic retinopathy, glaucoma and cardiovascular risk from colour fundus photographs. The register notes that the Australian configuration of Assure Plus excludes proliferative diabetic retinopathy and neovascular age-related macular degeneration.

None of the four is described on the register as deciding without a clinician. Each entry names the health professional it supports. The four FDA-cleared systems are not on the list. A name search of the Australian Register of Therapeutic Goods (ARTG) on 30 September 2026 found no entry for any of them ^{[19, 20]}. A name search cannot rule out an entry under a different product name or sponsor.

The one documented Australian programme, Lions Outback Vision's van in the Pilbara, ran Google's model with a remote clinician supervising by telehealth ^[8]. The first issue covers that programme, the rest of the list, and the exemption rule that changed in January 2026.

Conflict of interest, funding and how this was written

I own OptoGuide Pty Ltd, which sells OptoGuide, a clinical decision support product for optometrists. Its clinical recommendations come from fixed rules, and no AI model runs when a clinician uses it. I used AI tools to write its code, to draft its learning cases and to generate its clinical illustrations, which are labelled in the product as AI-generated. It has no ARTG entry. On 30 September 2026 I notified the TGA that it is supplied as exempt clinical decision support software, which is my own determination and not a ruling. It is live in Australia only. Versions for other countries have been researched and are not released. It has no published evaluation, no trial data and no outcome data, so every point in this issue about the gap between a pivotal trial and a clinic applies to it.

No one funded this issue. I have no financial relationship with Digital Diagnostics, Eyenuk, AEYE Health, iHealthscreen, Lions Outback Vision, Google, or any other maker or operator of a diabetic retinopathy system named here. I checked every figure against the paper or the register it is cited to, and the errors are mine. This series will cover the field without promoting OptoGuide, and where a finding bears on rule-based decision support I will say so.

The next issue covers the rules for Australian optometrists using AI: the guidance from the Australian Health Practitioner Regulation Agency (Ahpra) and the Code of conduct, the TGA's treatment of software as a medical device, the privacy obligations, including the automated decision-making transparency requirement that starts on 10 December 2026, and where liability sits when a tool is wrong.