Background/Purpose: Musculoskeletal ultrasound (US) is an imaging modality that is more sensitive in identifying synovitis due to inflammatory arthritis than clinical examination. Little is known about interrater reliability in rheumatologists trained in sonography for evaluating static rheumatoid arthritis US images from a musculoskeletal ultrasound sonographer. Our aim was to study the interrater agreement for hand ultrasounds for patients with findings for inflammatory arthritis in the hand.

Methods: A series of hand US for inflammatory arthritis was collected retrospectively from the University of Rochester Rheumatology Practice ultrasound bank. Hand US image series from adult patients ( > 18 years old) with a clinical diagnosis of seropositive rheumatoid arthritis referred for hand ultrasound in 2019 with reported symptoms of hand swelling or pain were included in this study. Image exclusion criteria were seronegative rheumatoid arthritis or another comorbid inflammatory arthritis diagnosis that could confound interpretation. Data collection continued until 30 eligible US image series were obtained. Participating rheumatologists were asked to complete a survey to score synovitis and tenosynovitis with Grey scale (GS) and Doppler (PD) in static images of at least 9 different joints and 4 flexor tendons of the hand and wrist. Reporting rheumatologists were asked to indicate the presence or absence of erosions, tenosynovitis, and to conclude whether inflammatory arthritis was present or absent based on the image series. Interrater agreement was evaluated using Gwet’s AC1, a statistic similar to Cohen’s Kappa. All analyses were done with Stata/MP 15.1 for Windows.

Results: Six rheumatologists with a mean (SD) of 7.5 (5.6) years reading US evaluated 15 patients’ hand and wrist US images and a subset of 4 rheumatologists evaluated an additional 15 patients (for a total of 30 image series) as a sensitivity analysis. All rheumatologists reported formal training in US. In 48 questions related to the US for each patient, the median Gwet’s AC1 was 0.78 with an interquartile range of [0.68, 0.85]. The question, “Is inflammatory arthritis present on these US findings?”, the responses were unanimous in 10 out of 15 cases. In 2 cases, 1 rater was discordant (e.g., 1 yes and 5 no responses) and in 3 cases, 2 raters were discordant: Gwet’s AC1 = 0.80; 95% CI (0.57, 1.00). The results for this question were similar in the sensitivity analysis as 4 raters were unanimous in 19 out of 30 images and Gwet’s AC1 = 0.67; 95% CI (0.46, 0.88).

Conclusion: While the interrater agreement among the rheumatologists was good (Gwet’s AC1 > 0.6) on the many US survey questions, the rheumatologists had different conclusions on the presence of inflammatory arthritis in some cases. Further evaluation with a consensus based scoring exercise among ultrasound trained rheumatologists based on OMERACT US definitions may improve agreement and also impact patient care.

« Back to ACR Convergence 2022

ACR Meeting Abstracts - https://acrabstracts.org/abstract/interrater-agreement-of-hand-ultrasounds-in-inflammatory-arthritis/