Background/Purpose: To utilize electronic health records (EHR) to study SLE, phenotypic algorithms are needed to accurately identify these patients. We aimed to generate an EHR algorithm for SLE using machine learning, which allows the data to inform algorithmic features, with the primary goal of optimizing the positive predictive value (PPV). We also aimed to compare this algorithm with the performance of published rule-based algorithms (Barnado et al. Arthritis Care Res 2017) that pre-specify combinations of ICD-9 codes, medications and laboratory tests in our EHR.

Methods: We randomly selected 400 subjects with ≥1 SLE ICD-9 code (710.0) from a large, academic medical system EHR, and two rheumatologists identified gold standard cases of definite and probable SLE. Subjects meeting 1997 ACR or 2012 SLICC Classification Criteria for SLE were classified as definite SLE; those with partial, usually 3 criteria, considered to have likely SLE by the treating rheumatologist and reviewers were defined as probable SLE. We divided subjects into a training set (N=200) and validation set (N=200). We extracted codified and narrative concepts using natural language processing (NLP) from the training set and generated algorithms using penalized logistic regression (LASSO) to classify subjects with definite or definite/probable SLE. Algorithms were applied to the validation set using the original case definition and validated externally at the institution where the rule-based algorithms were developed (N=175) using a more liberal definition of specialist-reported SLE diagnosis. We also applied published rule-based algorithms to our training set to assess portability.

Results: In the combined training and validation cohorts (N=200 each), 29% had definite SLE and 41% had definite/probable SLE. Using machine learning methods, our codified data algorithm had a PPV of 90% for definite SLE at 97% specificity and 64% sensitivity (Table 1). For definite/probable SLE, the PPV was 92% at 97% specificity and 47% sensitivity. Models with NLP data performed similarly. In the external cohort validation, the codified definite/probable SLE algorithm had 95% PPV, 98% specificity, and 13% sensitivity. The PPVs of rule-based algorithms were <50% for definite SLE and ≤65% for definite/probable SLE in our EHR (Table 1).

Conclusion: Our final machine learning SLE phenotype algorithms performed well in our EHR and had high PPV but lower sensitivity when externally validated in a cohort that did not require ACR/SLICC criteria to define cases. Rule-based SLE phenotype algorithms did not perform as well in our EHR likely because of these differences in case definitions and variations in clinical practice, medication use, laboratory tests, billing and documentation across EHRs. Unique EHR characteristics, case definitions, and research goals must be considered when applying algorithms to identify SLE patients in EHRs.