Electronic Health Record-Based Machine Learning Model for Predicting Disease Activity in Patients with Rheumatoid Arthritis

Zelin Yun¹, Chun Li² and Zhanguo Li², ¹Peking University People's Hospital, Beijing, China, ²Peking University People’s Hospital, Beijing, China

Meeting: ACR Convergence 2023

Keywords: Cohort Study, Outcome measures, rheumatoid arthritis

Session Information

Date: Sunday, November 12, 2023

Title: (0380–0422) RA – Diagnosis, Manifestations, and Outcomes Poster I

Session Type: Poster Session A

Session Time: 9:00AM-11:00AM

Background/Purpose: Rheumatoid arthritis (RA) is one of the most prevalent chronic inflammatory diseases in which synovitis gradually leads to polyarthritis and joint destruction. Poor response to initial therapy may contribute to progressive disease activity, high costs, and side effects. Given the explosive growth in the number of electronic health records (EHRs), we constructed machine learning models to predict disease activity and assess the effectiveness of treatment prior to the actual start of treatment.

Methods: This multicenter, prognostic study included 1864 patients with RA from 5 tertiary hospitals in China. Electronic health records (EHRs) were collected from 2016 to 2021. The Peking University People’s Hospital (PKUPH) cohort was used for training and internal validation. The multicenter cohort was used for external validation. Multi-dimensional variables arranged sequentially (baseline, 3 months, and 6 months after treatment), including demographic features, laboratory tests, and medication history information, were involved in machine-learning model construction. Different algorithms were selected to predict disease activity 6 months after treatment. External validation involved multicenter participants. The Disease Activity Score in 28 joints-Erythrocyte Sedimentation Rate (DAS28-ESR) ≤ 2.6 was categorized as the primary outcome.

Results: A total of 1629 patients from Peking University People’s Hospital (PKUPH) and 235 multicenter patients were included (number [%] female, 1353 [83.1] vs 197 [84.5]). There were significant differences in disease activity at baseline between the PKUPH and multicenter cohorts at baseline (median [IQRs] DAS28-ESR, 5.4 [4.8, 6.2] vs 6.1 [5.5, 6.7]). For the primary outcome, the best-performing predictive model attained an accuracy of 96.8% and an area under the receiver operating characteristic curve (AUROC) of 0.984 [sensitivity of 98.9%, specificity of 85.2%, positive predictive value (PPV) of 96.9%, and negative predictive value (NPV) of 94.4%]. Notably, in external validation, the best accuracy of 92.7% and AUROC of 0.977 were achieved. We also tested the performance of machine learning models with the secondary outcome (DAS28-ESR >5.1 for high disease activity, >3.2 to ≤5.1 for moderate disease activity, >2.6 to ≤3.2 for low disease activity, ≤2.6 for remission). The DNN model displayed an accuracy of 70.4% and an AUROC of 0.898, which is acceptable.

Conclusion: EHRs including multiple, sequential time nodes can be used to construct and improve machine learning models. Deep learning model can accurately predict the effectiveness of treatment within a moderate sample size. Constructing machine learning models to predict disease activity in patients with rheumatoid arthritis had significant benefits across distinct hospitals.

Table 1. The baseline characteristics of the study population
Data that are not normally distributed are presented as the medians and interquartile ranges (IQRs) unless otherwise indicated. CRP=C-reactive protein; DAS28=disease activity score using 28 joint counts; ESR=erythrocyte sedimentation rate; TJC28=tender joint count (of 28); SJC28=swollen joint count (of 28); PtGA=patient global assessment of disease activity; RF=rheumatoid factor; Anti-CCP=anti-cyclic citrullinated peptide.
* indicates a significant difference.

Figure 1. Evaluation of predictive machine learning models using the area under the receiver operating characteristic curve for primary outcome. The area under the receiver operating characteristic curve (AUROC) of machine learning (ML) models is displayed. Different colors were used to represent ML models. The results were attained from tests in the Peking University People’s Hospital (PKUPH) cohort. DNN=deep neural network; SVM=support vector machine; AdaBoost=adaptive boosting.

Figure 2. Evaluation of predictive machine learning models using the area under the receiver operating characteristic curve for external validation. The area under the receiver operating characteristic curve (AUROC) of machine learning (ML) models is displayed. Different colors were used to represent ML models. The results were attained from tests in the Peking University People’s Hospital (PKUPH) cohort. DNN=deep neural network; SVM=support vector machine; AdaBoost=adaptive boosting.

Disclosures: Z. Yun: None; C. Li: None; Z. Li: None.

To cite this abstract in AMA style:

Yun Z, Li C, Li Z. Electronic Health Record-Based Machine Learning Model for Predicting Disease Activity in Patients with Rheumatoid Arthritis [abstract]. Arthritis Rheumatol. 2023; 75 (suppl 9). https://acrabstracts.org/abstract/electronic-health-record-based-machine-learning-model-for-predicting-disease-activity-in-patients-with-rheumatoid-arthritis/. Accessed .

« Back to ACR Convergence 2023

ACR Meeting Abstracts - https://acrabstracts.org/abstract/electronic-health-record-based-machine-learning-model-for-predicting-disease-activity-in-patients-with-rheumatoid-arthritis/