ACR Meeting Abstracts

ACR Meeting Abstracts

  • Meetings
    • ACR Convergence 2024
    • ACR Convergence 2023
    • 2023 ACR/ARP PRSYM
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • ACR Meetings

Abstract Number: 0387

Absolute Agreement and Inter-rater Reliability of a Web-based Standardized Scoring Tool for Magnetic Resonance Images from Children with Chronic Nonbacterial Osteomyelitis (CNO) Amongst Radiologists

Farzana Nuruzzaman1, T. Shawn Sato2, Jennifer Stimec3, Ramesh S. Iyer4, Andrew Carbert5, Joel Paschke5, Lauren Potts6, Xiaoyue Zhang7, Walter Maksymowych8, Polly Ferguson9 and Yongdong (Dan) Zhao10, and CARRA CRMO Workgroup, 1Stony Brook Children's Hospital, Stony Brook, NY, 2University of Iowa, Iowa City, 3Hospital for Sick Children, Toronto, ON, Canada, 4Seattle Children's Hospital, Seattle, WA, 5CARE Arthritis, Edmonton, AB, Canada, 6Artist, Long Beach, CA, 7Renaissance School of Medicine at Stony Brook University, Stony Brook, NY, 8University of Alberta, Edmonton, AB, Canada, 9University of Iowa Carver College of Medicine, Iowa City, IA, 10University of Washington, Redmond, WA

Meeting: ACR Convergence 2024

Keywords: Autoinflammatory diseases, Magnetic resonance imaging (MRI), Measurement Instrument, Pediatric rheumatology

  • Tweet
  • Email
  • Print
Session Information

Date: Saturday, November 16, 2024

Title: Pediatric Rheumatology – Clinical Poster I

Session Type: Poster Session A

Session Time: 10:30AM-12:30PM

Background/Purpose: The web-based ChRonic nonbacterial Osteomyelitis Magnetic Resonance Imaging Scoring (CROMRIS) portal was developed to assess specific features of bone and soft tissue inflammation in MRI of patients with CNO in a user-friendly interface1. Primary objective of this study is to evaluate the absolute agreement of components of and summary CROMRIS scores at each body site, and the inter-rater reliability across raters.

Methods: A reliability module including 29 de-identified whole body (WB) MRI scans (coronal images of the entire body and sagittal images of entire spine, acquired with short tau inversion recovery [STIR] sequences without contrast) from children with CNO and normal controls was created. Three pediatric radiologists (TSS, RSI, JS), all with >10 years of expertise in pediatric musculoskeletal imaging independently scored these cases directly on the web platform. Prior to scoring, all the readers reviewed a training atlas and video. Readers scored individual bone units to indicate presence of: bone marrow hyperintensity (HI) on STIR images (score 0-1); soft tissue/periosteal HI of surrounding tissue (score 0-1); bony expansion (score 0-1), and quantify the signal size of CNO lesion (score 1-3 defined as < 25%, 25-50% or >50% of estimated volume, respectively). The sum of these parameters for lesions detected on fluid-sensitive sequences was the CROMIS Activity Index (maximum score 722).  Pairwise comparisons of the specific parameters and CROMRIS Activity Index were made. Absolute agreement was defined when the summary score difference between any 2 raters was ≤ 2. For specific variables (bone marrow HI, signal size, soft tissue/periosteal HI, bony expansion), the absolute agreement was defined as all the raters assigning the exact same score. Inter-rater reliability for abnormal signal and severity were assessed using free-marginal kappa (k) statistics.

Results: Among all bone sites, the prevalence of abnormalities were the highest at the tibia (59%), feet (41%), femur (38%), and pelvis (34%) (Figure 1). The proportion of absolute agreement of CROMRIS Activity Index scores across 3 raters was greater than 70% in most bone sites (Figure 2). Mean kappa of each category of bones was near or >0.5 demonstrating moderate inter-rater reliability amongst radiologists for most bone sites (Figure 3), except in lumbar spine, ulna, patella, mandible, sternum, scapula, and sacroiliac joints, which had mean k scores ranging from -0.04 to 0.21, correlating with sites of lowest prevalence of abnormalities in this module.

Conclusion: The web-based CROMRIS portal shows moderate agreement of CROMRIS Activity Index total scores amongst experienced radiologists after self-guided learning of atlas and video. A consensus-based set of scans for reader calibration will be developed to enhance inter-rater reliability for clinical trials.

The authors wish to acknowledge CARRA, CARE Arthritis and the ongoing Arthritis Foundation financial support of CARRA

  1. Nuruzzaman F, et al. Development and Usability Testing of Web-based Standardized Scoring Tool for Magnetic Resonance Images from Children with Chronic Nonbacterial Osteomyelitis (CNO) [abstract]. Arthritis Rheumatol. 2023; 75 (suppl 4).

Supporting image 1

Figure 1. Prevalence of abnormalities at each body site based on Bone Marrow hyperintensity (HI), Soft Tissue/Periosteal HI, and Bony Expansion scores among 29 cases

Supporting image 2

Figure 2. Absolute agreement of CROMRIS Activity Index across 3 raters at each body site among 29 cases

Supporting image 3

Figure 3. Inter-rater reliability across 3 raters of CROMRIS Activity Index across 3 raters at each body site based on Fleiss’ Kappa


Disclosures: F. Nuruzzaman: None; T. Sato: None; J. Stimec: None; R. Iyer: None; A. Carbert: None; J. Paschke: CARE Arthritis Limited, 3; L. Potts: None; X. Zhang: None; W. Maksymowych: AbbVie, 2, 5, 6, Boehringer Ingelheim, 2, 6, Bristol Myers Squibb (BMS), 2, 6, CARE Arthritis Limited, 4, Celgene, 2, 6, Eli Lilly, 2, 6, Galapagos, 2, 5, 6, Janssen, 2, 5, 6, Novartis, 2, 5, 6, Pfizer, 2, 5, 6, UCB Pharma, 2, 5, 6; P. Ferguson: None; Y. Zhao: Bristol-Myers Squibb(BMS), 5.

To cite this abstract in AMA style:

Nuruzzaman F, Sato T, Stimec J, Iyer R, Carbert A, Paschke J, Potts L, Zhang X, Maksymowych W, Ferguson P, Zhao Y. Absolute Agreement and Inter-rater Reliability of a Web-based Standardized Scoring Tool for Magnetic Resonance Images from Children with Chronic Nonbacterial Osteomyelitis (CNO) Amongst Radiologists [abstract]. Arthritis Rheumatol. 2024; 76 (suppl 9). https://acrabstracts.org/abstract/absolute-agreement-and-inter-rater-reliability-of-a-web-based-standardized-scoring-tool-for-magnetic-resonance-images-from-children-with-chronic-nonbacterial-osteomyelitis-cno-amongst-radiologists/. Accessed .
  • Tweet
  • Email
  • Print

« Back to ACR Convergence 2024

ACR Meeting Abstracts - https://acrabstracts.org/abstract/absolute-agreement-and-inter-rater-reliability-of-a-web-based-standardized-scoring-tool-for-magnetic-resonance-images-from-children-with-chronic-nonbacterial-osteomyelitis-cno-amongst-radiologists/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

All abstracts accepted to ACR Convergence are under media embargo once the ACR has notified presenters of their abstract’s acceptance. They may be presented at other meetings or published as manuscripts after this time but should not be discussed in non-scholarly venues or outlets. The following embargo policies are strictly enforced by the ACR.

Accepted abstracts are made available to the public online in advance of the meeting and are published in a special online supplement of our scientific journal, Arthritis & Rheumatology. Information contained in those abstracts may not be released until the abstracts appear online. In an exception to the media embargo, academic institutions, private organizations, and companies with products whose value may be influenced by information contained in an abstract may issue a press release to coincide with the availability of an ACR abstract on the ACR website. However, the ACR continues to require that information that goes beyond that contained in the abstract (e.g., discussion of the abstract done as part of editorial news coverage) is under media embargo until 10:00 AM ET on November 14, 2024. Journalists with access to embargoed information cannot release articles or editorial news coverage before this time. Editorial news coverage is considered original articles/videos developed by employed journalists to report facts, commentary, and subject matter expert quotes in a narrative form using a variety of sources (e.g., research, announcements, press releases, events, etc.).

Violation of this policy may result in the abstract being withdrawn from the meeting and other measures deemed appropriate. Authors are responsible for notifying colleagues, institutions, communications firms, and all other stakeholders related to the development or promotion of the abstract about this policy. If you have questions about the ACR abstract embargo policy, please contact ACR abstracts staff at [email protected].

Wiley

  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences

© Copyright 2025 American College of Rheumatology