Performance Analysis of a Deep Learning Algorithm to Detect Positive SIJ MRI According to the ASAS Definition in axSpA Patients

Joeri Nicolaes¹, Evi Tselenti², Theodore Aouad³, Clementina López Medina⁴, Antoine Feydy⁵, Hugues Talbot³, Bengt Hoepken⁶, Natasha De Peyrecave⁷ and Maxime Dougados⁸, ¹KU Leuven, Department of Electrical Engineering (ESAT), Center for Processing Speech and Images; UCB Pharma, Anderlecht, Belgium, ²Veramed, London, United Kingdom, ³CentraleSupélec, Universite Paris-Saclay, Inria, Gif-sur-Yvette, France, ⁴Rheumatology Department, Cochin Hospital; INSERM (U1153): Clinical Epidemiology and Biostatistics, University of Paris; Rheumatology Department, Reina Sofia Hospital, Cordoba / IMIBIC / University of Cordoba, Cordoba, Spain, ⁵INSERM (U1153): Clinical Epidemiology and Biostatistics, University of Paris; Radiology Department, Cochin Hospital, Paris, France, ⁶UCB Pharma, Monheim, Germany, ⁷UCB Pharma, Brussels, Belgium, ⁸Rheumatology Department, Cochin Hospital; INSERM (U1153): Clinical Epidemiology and Biostatistics, University of Paris, Paris, France

Meeting: ACR Convergence 2023

Keywords: Imaging, Magnetic resonance imaging (MRI), spondyloarthritis

Session Information

Date: Tuesday, November 14, 2023

Title: (1862–1894) Imaging of Rheumatic Diseases Poster II

Session Type: Poster Session C

Session Time: 9:00AM-11:00AM

Background/Purpose: Magnetic resonance imaging (MRI) of the sacroiliac joints (SIJ) is an essential tool in the evaluation of patients with axial spondyloarthritis (axSpA). In-depth knowledge of characteristic MRI lesions and their definitions, as well as reliability of identification and scoring, varies amongst general radiologists and rheumatologists.¹ A deep learning algorithm was developed to detect the presence of inflammation in SIJ MRI (MRI+) scans with promising results.² The aim of this diagnostic performance study was to assess the ability of a deep learning algorithm to identify MRI+ scans in a study cohort of axSpA patients.

Methods: 731 baseline SIJ MRI scans were collected from two prospective randomized controlled trial cohorts in patients with non-radiographic (nr-) and radiographic (r-) axSpA (RAPID-axSpA [NCT01087762] and C-OPTIMISE [NCT02505542])^3,4 and were centrally evaluated by two expert readers (and adjudicator in case of disagreement) for the presence of inflammation by the 2009 Assessment in SpondyloArthritis international Society (ASAS) definition.⁵ The MRI scans were processed by the previously trained deep learning algorithm,² blinded to clinical information and central expert readings. Performance evaluation included sensitivity, specificity, positive and negative predictive values (PPV and NPV), Cohen’s Kappa and the absolute agreement to assess the agreement between the deep learning algorithm and the human readers for the classification of MRI-SIJ scans. Bootstrapping was used to construct the 95% confidence interval (CI).

Results: Pooling the patients from RAPID-axSpA (n=152) and C-OPTIMISE (n=579) yielded a validation set of 731 patients (mean age: 34.2 years, SD: 8.6; 69.1% male) of which 44.6% were patients with nr-axSpA and 59.6% were MRI+ as per central readings. Comparing the trained algorithm with the human central readings for the classification of MRI+/MRI– on the pooled validation set yielded a sensitivity of 70% (95% CI: 66–73%), specificity of 81% (95% CI: 78–84%), PPV of 84% (95% CI: 82–87%), NPV of 64% (95% CI: 61–68%), Cohen’s kappa of 0.49 (95% CI: 0.43–0.55), and absolute agreement of 74% (95% CI: 72–77%; Table).

Conclusion: A previously trained deep learning algorithm enabled acceptable detection of the presence of inflammation according to the 2009 ASAS MRI definition in axSpA patients from two clinical trials. This suggests that an MRI+ detection algorithm has the potential to support clinicians in identifying axSpA patients.

References: 1. Bennett AN. J Rheumatol 2017;44(6):780–5; 2. Aouad T. Proc Int Conf Image Proc 2022;3351–5; 3. van der Heijde. Rheumatology. 2017;56(9):1498–1509; 4. Landewé RB. Ann Rheum Dis. 2020;79(7):920–28; 5. Rudwaleit M. Ann Rheum Dis 2009;68(6):777–83.

Table. Performance results comparing the algorithm and the human readers for the classification of MRI-SIJ scans. The metric values are point estimate (95% CI)

Disclosures: J. Nicolaes: UCB Pharma, 3, 12, Shareholder; E. Tselenti: UCB Pharma, 2, Veramed, 3; T. Aouad: None; C. López Medina: AbbVie, 5, 6, Eli Lilly, 2, 5, 6, Janssen, 6, MSD, 6, Novartis, 2, 5, 6, UCB Pharma, 2, 5, 6; A. Feydy: Guerbet, 2; H. Talbot: None; B. Hoepken: UCB Pharma, 3, 12, Shareholder; N. De Peyrecave: UCB Pharma, 3; M. Dougados: AbbVie, 2, 5, Bristol Myers Squibb, 2, 5, Eli Lilly, 2, 5, MSD, 2, 5, Novartis, 2, 5, Pfizer Inc, 2, 5, Roche, 2, 5, UCB, 2, 5.

To cite this abstract in AMA style:

Nicolaes J, Tselenti E, Aouad T, López Medina C, Feydy A, Talbot H, Hoepken B, De Peyrecave N, Dougados M. Performance Analysis of a Deep Learning Algorithm to Detect Positive SIJ MRI According to the ASAS Definition in axSpA Patients [abstract]. Arthritis Rheumatol. 2023; 75 (suppl 9). https://acrabstracts.org/abstract/performance-analysis-of-a-deep-learning-algorithm-to-detect-positive-sij-mri-according-to-the-asas-definition-in-axspa-patients/. Accessed .

« Back to ACR Convergence 2023

ACR Meeting Abstracts - https://acrabstracts.org/abstract/performance-analysis-of-a-deep-learning-algorithm-to-detect-positive-sij-mri-according-to-the-asas-definition-in-axspa-patients/