AI predictive ability benefits from free text data in EHRs

Reviewed by Sophia Y. Wang, MD

Advances in the use of artificial intelligence (AI) may be useful in treating patients with glaucoma because of the unpredictable nature of the disease.

Machine learning models that use electronic health records (EHRs) have tried to predict the disease course, but it was not clear whether adding text data from clinical notes would be helpful in any way, according to Sophia Y. Wang, MD, an assistant professor of ophthalmology at Byers Eye Institute, Stanford University in California.

“There are so many clinical details that are stored in the free-text clinical progress notes that are very difficult to access and compute over,” she said.

Related: Electronic health records: Moving beyond 'just a requirement'

However, Wang and Benjamin Tseng, AB, also from Byers Eye Institute, did just that.

They found that the addition of textual clinical notes contained highly valuable information that helped better predict the course of glaucoma that may ultimately require surgery in patients.

How they did it
The investigators used the clinical notes of glaucoma patients obtained from the EHRs. They included demographic data, diagnosis codes, surgical history, information about intraocular pressure, visual acuity, and central corneal thickness.

Then, “the words from the patients’ 120 days of notes were mapped into the ophthalmology domain-specific neural work embeddings, a natural language processing technique in which the word meanings are encoded in the vector space. …Once the words are mapped to the numeric vectors, they can more easily be used in deep-learning predictive models,” Wang explained.

She and Tseng compared this dual-input model with one that used only structured input data and one that contained only text inputs.

Related: Seven ways to 'hack' your EHR for the best customized alerts

Patients were included who had (n = 748) and had not (n = 3764) undergone an incisional glaucoma surgery. All patient data were from the Stanford University repository data from 2008 to 2020.

What they found
Wang noted that deep learning models developed using EHR data can surpass human performance in predicting whether glaucoma patients will need surgery.

“Incorporating free-text data appeared to improve performance compared with utilizing only structured inputs,” she said.

Wang showed the receiver operating curve results for the combination model (area = 0.731) were slightly better than those of the text curve (area = 0.697) and the structured curve (area = 0.658).

All the models outperformed a review of 300 charts, the notes by an ophthalmologist to determine the human-level predictive value of the charts, and notes for determining which patients would ultimately need surgery.

Related: EHR-linked glaucoma medication reminder may improve adherence

The precision recall curves showed that the text model (area = 0.431) showed slightly better precision recall than the structured (area = 0.284) and combination (area = 0.392) models.

“Using word embeddings to represent clinical notes, deep learning models could predict whether glaucoma patients would need a future glaucoma surgery at a performance level better than an ophthalmologist’s review of the same notes,” Wang said.

She also noted that predictive models can be helpful in clinical decision support or for automatically identifying high-risk patients for clinical trials.

However, she offered the caveat that more work needs to be done to improve the performance before the models are deployed for clinical use.

Related: Increasing your practice’s reach, relevance, and revenue with clinical trials

This work is a first step, Wang pointed out. Future work may include collaborating with other clinical centers, incorporating imaging into the predictive models, and using more sophisticated representation methods for text, such as transformer-based models.

Additionally, named entity recognition systems to produce features from the text and investigations of performance in subgroups of patients based on physician and race or ethnicity may also be used in future work.

--

Sophia Y. Wang, MD
e:sywang@stanford.edu
This article is adapted from Wang’s presentation at the American Glaucoma Society’s 2021 virtual annual meeting. She has no financial interest in this subject matter.