Background/Purpose: Magnetic resonance imaging (MRI) offers a non-invasive and objective method of early diagnosis and classification, monitoring disease burden and treatment response for patients (pts) with axial spondyloarthritis (axSpA).¹ Numerous scoring schemes such as the Ankylosing Spondylitis (AS) Spine MRI Activity (ASspiMRIa) score are available for the quantitative assessment of MRI, but are subject to intra- and inter-rater variability, are labor intensive and costly. Nevertheless, quantification of MRI changes has become an important tool to demonstrate treatment success of biologic drugs in axSpA. In this study, we evaluate the performance of machine learning (ML) based software for the automated Berlin grading of spinal MRI bone marrow edema in pts with AS and compare with expert scoring.

Methods: Fully automated ML software (Figure) was developed to detect and label 23 vertebrae, define vertebral units (VU) as per the Berlin modification of the ASspiMRIa score, and score each VU as either 0 (score of 0) or 1 (score of 1, 2 or 3). The ML algorithm was based on the previously developed SpineNet software.² Analysis included 108 pts from the secukinumab MEASURE 1 study³, in which imaging was done using T1 and STIR sagittal MRI at baseline and Weeks 16, 52, 104, 156 and 208. Two expert readers (R1 and R2), blinded to treatment and visit, evaluated all images by Berlin-modified ASspiMRIa score. The scores from both readers were binned or categorized into two groups: 0 vs 1, 2, or 3. As a result of multiple pt time points and expert reading sessions, the complete dataset comprised of 10,988 VUs, (8,954 graded by R1, 9,010 graded by R2). Ten-way cross-validation at the VU level was used to train and validate the ML software. The dataset was split into 10 randomly selected subsets, ensuring that each pt appears in only one subset, after which 8 subsets were used for training the ML software, one was used to check for correct training, and one was used for validation. The process was repeated 10 times such that all 10 subsets were used for validation. Area-Under-the-Curve (AUC), accuracy weighted for the frequency of each category, sensitivity and specificity were calculated. Intra- and inter-reader accuracy were also calculated.

Results: Average AUC and accuracy of the software in relation to expert readers’ scores was 0.80 and 72% with a sensitivity of 0.72 and specificity of 0.72. The intra- and inter-reader agreement was 75% (R1), 83% (R2) and 73% respectively. Individual VU scoring of the software vs. readers are presented in the Table as a confusion matrix.

Conclusion: Automated scoring of MR images in AS pts was comparable to that of expert reader-based assessments. ML software has the potential to provide an automated guided-reading approach to scoring MR images, which may enable further clinical insights.

References:

1. Lukas C, et al. J Rheumatol. 2007;34:862-70.
2. Jamaludin A, et al. Eur Spine J. 2017;26:1374-83.
3. Baeten D, et al. N Engl J Med. 2015;373,2534-48.

Processing pipeline of automated Berlin scoring software

Confusion matrix between the software, R1 and R2

« Back to ACR Convergence 2020

ACR Meeting Abstracts - https://acrabstracts.org/abstract/machine-learning-based-berlin-scoring-of-magnetic-resonance-images-of-the-spine-in-patients-with-ankylosing-spondylitis-analysis-of-data-from-a-phase-3-trial-with-secukinumab/