ACR Meeting Abstracts

ACR Meeting Abstracts

Abstract Number: 1016

The Effect of Evaluation Variability At the Unit of Measurement On the Reliability of Omeract Ramris and Van Der Heijde-Modified Sharp Score

Ruben Tavares1, Naveen Parasu2, Karen Finlay3, Erik Jurriaans3, Hao Wu1, Karen A. Beattie4, Maggie Larche5, Lawrence E. Hart6, William G. Bensen7, Raja S. Bobba5, Alfred A. Cividino4, Colin E. Webber8, Jean-Eric Tarride9 and Jonathan D. Adachi10, 1McMaster University, Hamilton, ON, Canada, 2Radiology, Hamilton Health Sciences, Hamilton, ON, 3Radiology, Hamilton Health Sciences, Hamilton, ON, Canada, 4Medicine, McMaster University, Hamilton, ON, Canada, 5Rheumatology, McMaster University, Hamilton, ON, Canada, 6Medicine (Rheumatology), St. Joseph's Health Care, Hamilton, ON, Canada, 7Department of Medicine, Division of Rheumatology, St. Joseph's Hospital and McMaster University, Hamilton, ON, Canada, 8Department of Nuclear Medicine, Hamilton Health Sciences, Hamilton, ON, Canada, 9Clinical Epidemiology and Biostatistics (CE&B) and Economics, Programs for Assessment of Technology in Health (PATH) Research Institute, Hamilton, ON, Canada, 10St Joseph's Healthcare Hamilton, Hamilton, ON, Canada

Meeting: 2012 ACR/ARHP Annual Meeting

Keywords: , , ,

Session Information

Title: Imaging of Rheumatic Diseases: Magnetic Resonance Imaging, Computed Tomography and X-ray

Session Type: Abstract Submissions (ACR)

Background/Purpose: The Outcome Measures in Rheumatology (OMERACT) Rheumatoid Arthritis Magnetic Resonance Imaging Score (RAMRIS) and van der Heijde-modified Sharp score (vdHSS) are recognized semi-quantitative measures for the evaluation of rheumatoid arthritis (RA) disease progression on magnetic resonance imaging (MRI) and radiography (X-ray), respectively.  The smallest detectable difference (SDD) is used to quantify the reliability between ratings of status scores in scale units.  To date, the SDD has been evaluated at the overall or component feature score levels of analysis thereby ignoring variability at the unit of measurement.  The objective of this study was to determine the reliability of RAMRIS and vdHSS at the unit of measurement across four radiologists and to compare it to the conventional approach.

Methods: A paired, cross-sectional study of RA patients with varying symptom duration (mean 6.8 years (SD 6.4) for MRI, 7.6 years (SD 7.3) for X-ray) was conducted.  19 MR image sets of metacarpophalangeal joints (MCP) 2-5 and 9 X-ray image sets of both hands, wrists, and forefeet were each independently evaluated by 4 radiologists using RAMRIS and vdHSS, respectively.  Shrout and Fleiss fixed and random effects intra-class correlation coefficients (fICC and rICC, respectively), and SDD were calculated for overall and component feature scores, as well as at the unit of measurement.

Results: At the unit of measurement, the RAMRIS erosion, edema, and synovitis fICC were 0.71, 0.56, and 0.41, respectively.  The corresponding SDD values (rounded to scale unit) for erosion, edema, and synovitis were 2, 1 and 2.  Overall and component feature score reliability measures were dependent on the anatomy compared.  For single MCP 2-5 joint sets (i.e. per hands), the fICC for the overall RAMRIS and component feature subscore for erosion, edema, and synovitis were 0.66, 0.55, 0.60, and 0.39.  The corresponding SDD values were 11, 7, 4, and 5.  For X-ray, erosion and joint space narrowing (JSN) fICC at the unit of measurement were 0.61 and 0.69, and the associated SDD values were both 2.  For the hands, wrists and feet, the overall and component feature scores for erosion and JSN were 0.69, 0.60, and 0.85.  The corresponding SDD values were 39, 34, and 11.

Conclusion: The conventional approach to the calculation of reliability for overall or component feature scores fails to account for variability at the unit of measurement.  In order for reliability of composite measure scores to be valid, the assumption is made that the SDD at the unit of measurement is less than the scale unit.  The study findings suggest that this prerequisite assumption may be false.  The validity of literature evidence for the reliability of diagnostic imaging and composite measures in general is therefore questionable.

Disclosure:

R. Tavares,
None;

N. Parasu,
None;

K. Finlay,
None;

E. Jurriaans,
None;

H. Wu,
None;

K. A. Beattie,
None;

M. Larche,
None;

L. E. Hart,
None;

W. G. Bensen,
None;

R. S. Bobba,
None;

A. A. Cividino,
None;

C. E. Webber,
None;

J. E. Tarride,
None;

J. D. Adachi,
None.

« Back to 2012 ACR/ARHP Annual Meeting

ACR Meeting Abstracts - https://acrabstracts.org/abstract/the-effect-of-evaluation-variability-at-the-unit-of-measurement-on-the-reliability-of-omeract-ramris-and-van-der-heijde-modified-sharp-score/