Background/Purpose: Radiographic assessment of the sacroiliac joints (SIJ) according to the modified New York (mNY) criteria is essential in the classification of ankylosing spondylitis, but low to moderate inter-reader agreement and limited improvement by training have been reported. A web-based real-time iterative calibration (RETIC) module for scoring SIJ radiographs according to the mNY criteria has been developed by experienced readers to allow remote standardized training. In this study, we aimed to test the performance of this online RETIC training module in enhancing scoring proficiency with an expert reader as gold standard reference.

Methods: The RETIC module for scoring according to the mNY criteria consisted of 50 cases with radiographs of the SIJ at one timepoint with an integrated scoring interface with real time feedback to the reader’s grading compared to the expert readers’ grading. The mNY grade (0-4) is assessed for each joint. Reliability for fulfilling (mNY+) versus not fulfilling (mNY-) mNY criteria was assessed in real-time by kappa data being provided every 10 cases, until proficiency targets (kappa > 0.5) were achieved. In this study, participants (n=17) from the EuroSpA Imaging project were randomized to one of two reader training strategies (groups A and B) that comprised of 3 training steps, each followed by a reading exercise (Exercises 1-3; 25 patients per exercise). Training steps were as follows: Group A. 1. Review of original manuscripts describing the mNY method. 2. Review of PowerPoint summary of mNY method and video with grading examples PLUS completion of mNY_RETIC module. 3. Re-review of PowerPoint summary and video. Group B. Same 3-step strategy as group A except that the mNY_RETIC module was completed at step 3, i.e. immediately before Exercise 3. The reliability of scoring was compared to an expert radiologist (RL), using kappa statistics for positivity of mNY criteria and intraclass correlation coefficients (ICCs) for the grades of the mNY criteria.

Results: The mean agreement for all readers on fulfillment of the mNY criteria (mNY+ vs mNY-) with kappa was 0.69 in exercise 1, 0.72 in exercise 2 and 0.83 in exercise 3. Corresponding ICCs for mNY scoring 0-4 were 0.77, 0.77 and 0.82 in exercises 1, 2 and 3, respectively, see Table. For Reader Group A (performed RETIC module before exercise 2), kappas for exercises 1-3 were 0.65, 0.76 and 0.83, respectively and ICCs 0.71, 0.82 and 0.81 (exercise 3). Interestingly, the main increase in agreement occurred between Exercises 1 and 2, i.e. after the RETIC module was performed. For Reader group B (performed RETIC module before exercise 3), corresponding kappa’s were 0.71, 0.77 and 0.83 and ICCs 0.80, 0.72 and 0.80, for exercises 1-3, respectively. Similarly, agreements predominantly improved directly after the RETIC module was performed.

Conclusion: Agreement on scoring radiographs of the SIJ according to the mNY criteria was noticeably improved when using a systematic online training module and in general readers achieved very good agreement with experts implying that systematic training of readers before scoring of images can enhance scoring proficiency.

« Back to ACR Convergence 2022

ACR Meeting Abstracts - https://acrabstracts.org/abstract/increasing-the-proficiency-for-scoring-sacroiliac-joint-radiographs-according-to-the-modified-new-york-criteria-for-ankylosing-spondylitis-by-using-an-online-real-time-iterative-calibration-retic-mo/