Background/Purpose: The diagnostic role of Magnetic Resonance Imaging (MRI) for axial spondyloarthritis(axSpA) has grown since introducing the ASAS criteria for axSpA. Recent advances in machine learning (ML) haveshown promisein MRI interpretation for sacroiliitis detection in axSpA. Thisstudy aims to provide a systemic review and a summary estimate ofperformance indices of MLalgorithms in discriminating sacroiliitis in accordance with axSpAon MRI.

Methods: Embase, MEDLINE, CINAHL Complete, Web of Science, and IEEE Xplore Digital Library databases were used. The search was performed on 5 June 2023. Screening and extraction were performed independently by two authors according to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM). To summarize sensitivity and specificity, a bivariate random effects model was employed. The Summary Receiver Operating Characteristics (SROC) curve derived from parameters extracted from a given model is presented with summary points defined by 95% confidence and prediction regions. This graph can also derive the partial Area Under the Curve (pAUC). Risk of bias was assessed by the checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (PROBAST). The protocol was registered before the review started (CRD42023415446).

Results: A total of 1,986 studies were searched. Following the selection process, ten and seven studies were used for qualitative assessment and meta-analysis, respectively. The majority of studies were original articles, predominantly from European and Asian regions, and based on hospital cohorts. Also, half of the 10 studiesutilized deep learning, and three studiesconducted external validation. Nine studies used the discrimination of radiologic SpA features of ASAS as the ground truth. Seven validation or test datasets (274 positive and 375 negative sacroiliitis patients) were included in the meta-analysis. Its pooled sensitivity, specificity, and pAUC were 79% (95% CI: 70-86%), 80% (73-85%), and 0.79 respectively (Figure 1). Two studies conducted external validation and comparison between machine and human doctors. The Ensembling Mask regions with convolutional neural network algorithm’s accuracy achieved 92% (88-100%), and that of expert radiologists’ showed a range of 92-98%. The deep neural network algorithm’s accuracy from the other study showed 78% (69-85%), and that of expert and non-expert radiologists’ were an average of 94% (87-97%) and 86% (79-92%) respectability (Table). The summary of the quality appraisal tool (PROBAST) showed that more than half of the studies (six of 10 studies) were unclear and had a high risk of bias in the analysis domain (Figure 2).

Conclusion: ML algorithms for discriminating SpAfeatures in MRI have acceptable pooled sensitivity and specificity (around 80%). However, there are potential bias concerns in the validation process, and the number of studies comparing the accuracy of human experts and machines was small, limiting interpretation. An accuracy issue of machine leaning algorithms inSpA can be concluded in the future if well-designed and transparent reports that follow a guideline become more widely available.

« Back to ACR Convergence 2023

ACR Meeting Abstracts - https://acrabstracts.org/abstract/performance-of-machine-learning-algorithms-in-discriminating-spondyloarthritis-features-on-magnetic-resonance-imaging-a-systematic-review-and-meta-analysis/