Background/Purpose: To date, most of the models used to identify gout cases within large administrative databases have relied solely only on administrative billing codes. The positive predictive value (PPV) of these models ranged from 33-86%. Natural language processing (NLP) is a range of computational techniques for analyzing and representing naturally occurring written or oral text for the purpose of achieving human-like language processing for a range of tasks or applications. In this study we aimed to develop and validate an algorithm that accurately identifies gout patients within the Partners biobank database using both codified data and information from clinical text notes using NLP.

Methods: To create a gold-standard training set, a training set of 200 patients was created. Two rheumatologists reviewed the electric medical records of the 200 patients and classified them as having the disease (Y), probably having the disease (P), not having the disease (N) or unable to make a classification (U). We used the clinician-reviewed classifications to train models to predict the probability of a gout diagnosis or no gout on the basis of a logistic regression classifier with the adaptive least absolute shrinkage and selection operator (LASSO) procedure to select informative variables. We constructed three separate models to predict a diagnosis of gout in our partners biobank cohort- (1) model utilizing number of gout ICD-9 codes alone (ICD-9 model), (2) model comprising all codified variables including disease complications (codified model) (3) a combined model including both codified and NLP variables (combined model).

Results: The area under the curve (AUC) for the combined model was 0.901 (95% CI 0.830-0.972), with a sensitivity of 0.936 at a positive predictive value cut-off of 0.902. The AUC of the ICD-9 model was 0.721 (95% CI 0.617-0.825), while that of the codified model was 0.879 (95% CI 0.806-0.952). Addition of NLP narrative terms to our final model resulted in improving the sensitivity to 0.936 from 0.89, at the same PPV level of 0.902, thus resulting in improved identification of gout cases by 4.12%, compared to the codified model. On review of medical records from an additional random set of 50 patients each predicted to have gout by the combined model, 44 were correctly identified as having this diagnosis through chart review resulting in a positive predictive value of 88%.

Conclusion: Including narrative concepts from natural language processing improves the accuracy of EMR case-definition for gout while simultaneously identifying more subjects compared to models using codified data alone.

« Back to 2016 ACR/ARHP Annual Meeting

ACR Meeting Abstracts - https://acrabstracts.org/abstract/improving-predictive-value-of-gout-case-definitions-in-electric-medical-records-utilizing-natural-language-processing-a-novel-informatics-approach/