ACR Meeting Abstracts

ACR Meeting Abstracts

  • Meetings
    • ACR Convergence 2024
    • ACR Convergence 2023
    • 2023 ACR/ARP PRSYM
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • ACR Meetings

Abstract Number: 0745

Artificial Intelligence Models for Computer-Assisted Joint Detection and Sharp-van Der Heijde Score Prediction in Hand Radiographs from Patients with Rheumatoid Arthritis

Carol Hitchon1, Saqib Al Islam1, Daryl LX Fung1, Qian Liu1, Leann Lac1, Susan Bartlett2, Louis Bessette3, Gilles Boire4, Vivian Bykerk5, Glen Hazlewood6, Edward Keystone7, Janet Pope8, Orit Schieir9, Carter Thorne10, Diane Tin11, Marie-France Valois12, Désirée van der Heijde13, Canadian Early Arthritis Cohort (CATCH) Investigators14, Liam O'Neil1 and Pingzhao Hu15, 1University of Manitoba, Winnipeg, MB, Canada, 2McGill University, Montreal, QC, Canada, 3Centre de l'Ostéoporose et de Rhumatologie de Québec, Quebec City, QC, Canada, 4Université de Sherbrooke, Sherbrooke, QC, Canada, 5Hospital for Special Surgery, New York, NY, 6University of Calgary, Calgary, AB, Canada, 7Keystone Consulting Enterprises Inc., Toronto, ON, Canada, 8University of Western Ontario, London, ON, Canada, 9McGill University, Montréal, QC, Canada, 10Southlake Regional Health Centre, Newmarket, ON, Canada, 11The Arthritis Program Research Group, Newmarket, ON, Canada, 12McGill University, Pointe-Claire, QC, Canada, 13Department of Rheumatology, Leiden University Medical Center, Leiden, Netherlands, 14CATCH, Winnipeg, MB, Canada, 15Western University, London, ON, Canada

Meeting: ACR Convergence 2023

Keywords: Erosions, Measurement Instrument, rheumatoid arthritis, X-ray

  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
Session Information

Date: Sunday, November 12, 2023

Title: Abstracts: Imaging of Rheumatic Diseases

Session Type: Abstract Session

Session Time: 2:00PM-3:30PM

Background/Purpose: Radiographs are used to detect and monitor joint damage due to rheumatoid arthritis (RA).Scoring methods such as the Sharp van der Heijde tool (SVH) quantify joint space narrowing (JSN) and erosions. Obtaining SVH scores is time-consuming and requires expertise not always available. We aimed to develop and validate a deep learning system for automated detection and prediction of SVH scores in hand radiographs of patients with RA.

Methods: We used a convolutional neural network (CNN) based algorithm (You Only Look Once (YOLO)v5l6) trained on an object detection model (COCO-Common Objects in Context) to detect joints in 240 training and 89 test pediatric hand radiographs from the Radiologic Society of North America database. Radiographs were annotated by boxing and labeling the joints of interest: proximal interphalangeal, metacarpophalangeal, wrist, distal radius, distal ulna. Images were augmented using mix-up (mixing features of different images into one), mosaic (combining 4 training images into 1 image), rotation, translation, scaling, and shearing. The joint detection model was validated with 54 clinician-annotated radiographs from 4 adult RA patients followed for 9-13 years (10-12 images per patient) (joint detection gold standard). We applied a supervised vision transformer model (VTM) to predict each joint’s SVH erosion and JSN score. The VTM was validated using 2249 hand radiographs with clinician-assigned SVH scores from 381 RA patients from the Canadian Early Arthritis Cohort (SVH score gold standard). We applied techniques used to train highly class-imbalanced datasets including weighted sampling, stratified data-split, data augmentation and transfer learning (TL). As the joint detection model was trained to detect the whole wrist and we had clinician-assigned SVH scores for individual wrist joints, we trained a separate multi-task model to predict wrist joint scores from whole wrist images. The performance of the VTM to predict joint scores was compared to CNN-based EfficientNetV2 and MobileNetV3. Model accuracy for joint detection is reported as the F1-score (reflecting model precision and model recall) and mean absolute precision (mAP) for a range of Intersection-over-Union (IoU) measures reflecting the overlap between clinician and model assigned bounding boxes of detected joints. Accuracy for SVH score prediction is reported as root mean squared error (RMSE) and balanced accuracy.

Results: The joint detection model accurately identified target joints (pediatric data FI-score = 0.991, map0.1 = 0.993 with an IoU threshold of 0.1 ; adult data F1-score = 0.812, map0.1 = 0.871 (n=54) (Table 1). Applying TL improved the joint detection model’s mean precision by 0.05 over the COCO model. The VTM predicted JSN and erosion SVH scores with high accuracy (RMSE JSN 0.91, erosion 0.93). The multi-task models predicted SVH erosion and JSN scores of wrist joints with moderately high accuracy (0.6-0.91). EfficientNetV3 performed better for wrist joints (VTM vs EfficientNetV3 average difference 0.10) (Figure 1).

Conclusion: Automated deep learning systems accurately identify and predict joint damage in hand radiographs from patients with rheumatoid arthritis and may aid in monitoring joint damage.

Supporting image 1

Table: Accuracy of model for joint detection

Supporting image 2

Figure: Joint damage score prediction models. A) Models trained on all joints except wrist. The metrics reported were balanced accuracy, F1-Score and Root Mean Square Error (RMSE) predicting Joint Space Narrowing (JSN) and Erosion scores. B) Models were trained on wrist joints. Separate models trained to detect the erosion score and JSN scores. PIP=proximal interphalangeal, MCP=metacarpophalangeal, JSN= joint space narrowing, Vit= Vision Transformer, mc = metacarpal, nav = navicular, cmc = carpometacarpal


Disclosures: C. Hitchon: Astra Zeneca, 1, Pfizer, 5; S. Al Islam: None; D. Fung: None; Q. Liu: None; L. Lac: None; S. Bartlett: Janssen, 6, Merck/MSD, 2, 6, Novartis, 2, Organon, 1, 6, PROMIS Health Organization, 4, Sandoz, 2, 6; L. Bessette: AbbVie, 2, 5, 6, Amgen, 2, 5, 6, Bristol Myers Squibb, 2, 5, 6, Eli Lilly, 2, 5, 6, Fresenius Kabi, 2, 6, Gilead, 2, 5, 6, JAMP Pharma, 2, 5, 6, Janssen, 2, 5, 6, Novartis, 2, 5, 6, Organon, 2, 6, Pfizer, 2, 5, 6, Sandoz, 2, 6, Sanofi, 2, 5, 6, Teva, 2, 6, UCB, 2, 5, 6, UCBA, 5; G. Boire: Eli Lilly, 1, Janssen, 6, Organon, 1, Orimed Pharma, 1, 6, Otsuka, 1, Pfizer, 1, 5, Sandoz, 1, Teva, 1, Viatris, 1, 6; V. Bykerk: Abbvie, 2, BMS, 2, Pfizer, 2; G. Hazlewood: None; E. Keystone: AbbVie/Abbott, 2, 6, Amgen, 2, 6, celltrion, 2, 6, Eli Lilly, 2, 6, Fresenius Kabi, 2, 6, Pfizer, 2, 6, Samsung Bioepsis, 2, sandoz, 2, 6; J. Pope: AbbVie, 1, 2; O. Schieir: None; C. Thorne: Abbvie, 1, Biogen, 2, Nordic Pharma, 1, Pfizer, 1, 5, Roche, 1, Sandoz, 1, 2; D. Tin: None; M. Valois: None; D. van der Heijde: AbbVie, 2, Bayer, 2, BMS, 2, Eli Lilly, 2, Galapagos, 2, Gilead, 2, GSK, 2, Imaging Rheumatology BV, 12, Director, Janssen, 2, Novartis, 2, Pfizer, 2, Takeda, 2, UCB Pharma, 2; C. (CATCH) Investigators: AbbVie Corporation, 5, Amgen Canada, 5, Hoffman La Roche Limited, 5, Medexus Pharmaceuticals, 5, Organon Canada, 5, Pfizer Canada, 5, Sandoz Biopharmaceuticals Canada, 5; L. O'Neil: None; P. Hu: None.

To cite this abstract in AMA style:

Hitchon C, Al Islam S, Fung D, Liu Q, Lac L, Bartlett S, Bessette L, Boire G, Bykerk V, Hazlewood G, Keystone E, Pope J, Schieir O, Thorne C, Tin D, Valois M, van der Heijde D, (CATCH) Investigators C, O'Neil L, Hu P. Artificial Intelligence Models for Computer-Assisted Joint Detection and Sharp-van Der Heijde Score Prediction in Hand Radiographs from Patients with Rheumatoid Arthritis [abstract]. Arthritis Rheumatol. 2023; 75 (suppl 9). https://acrabstracts.org/abstract/artificial-intelligence-models-for-computer-assisted-joint-detection-and-sharp-van-der-heijde-score-prediction-in-hand-radiographs-from-patients-with-rheumatoid-arthritis/. Accessed .
  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print

« Back to ACR Convergence 2023

ACR Meeting Abstracts - https://acrabstracts.org/abstract/artificial-intelligence-models-for-computer-assisted-joint-detection-and-sharp-van-der-heijde-score-prediction-in-hand-radiographs-from-patients-with-rheumatoid-arthritis/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

All abstracts accepted to ACR Convergence are under media embargo once the ACR has notified presenters of their abstract’s acceptance. They may be presented at other meetings or published as manuscripts after this time but should not be discussed in non-scholarly venues or outlets. The following embargo policies are strictly enforced by the ACR.

Accepted abstracts are made available to the public online in advance of the meeting and are published in a special online supplement of our scientific journal, Arthritis & Rheumatology. Information contained in those abstracts may not be released until the abstracts appear online. In an exception to the media embargo, academic institutions, private organizations, and companies with products whose value may be influenced by information contained in an abstract may issue a press release to coincide with the availability of an ACR abstract on the ACR website. However, the ACR continues to require that information that goes beyond that contained in the abstract (e.g., discussion of the abstract done as part of editorial news coverage) is under media embargo until 10:00 AM ET on November 14, 2024. Journalists with access to embargoed information cannot release articles or editorial news coverage before this time. Editorial news coverage is considered original articles/videos developed by employed journalists to report facts, commentary, and subject matter expert quotes in a narrative form using a variety of sources (e.g., research, announcements, press releases, events, etc.).

Violation of this policy may result in the abstract being withdrawn from the meeting and other measures deemed appropriate. Authors are responsible for notifying colleagues, institutions, communications firms, and all other stakeholders related to the development or promotion of the abstract about this policy. If you have questions about the ACR abstract embargo policy, please contact ACR abstracts staff at [email protected].

Wiley

  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences

© Copyright 2025 American College of Rheumatology