Background/Purpose:

Digital
ulcers (DUs) are often a primary end point in SSc clinical trials, although the
reliability of rheumatologists grading DUs is poor to moderate at best. This is
of concern as DUs are often a primary outcome measure in SSc clinical trials.
DU assessment in recent trials has been based upon visual inspection alone,
which potentially misses ‘real-world’ clinical contextual information (e.g.
pain and discharge). Our aim was to investigate whether this clinical
information improves the reliability of rheumatologists grading DUs.

Methods:

80
photographs of a range of digital lesions were collected from patients with
SSc-spectrum disorders (mainly SSc). The following clinical information was
collected for each image:  duration of
the lesion (patient reported), pain associated with the lesion on a visual
analogue score (100 being most severe) and its temporal relationship (whether
the pain was less, the same or worse than a week ago), and the presence of
discharge (both patient reported and clinician observed). A custom web-based
interface was constructed to display the images and record the grading.
Rheumatologists were invited to participate through SSc-specialist networks and
were randomised to receive all images either with or without context. Lesions
were graded on an ordinal scale of severity: ‘no ulcer’, ‘inactive ulcer’ or
‘active ulcer’. Reliability was assessed using a weighted kappa coefficient,
with bootstrapping to generate estimates of confidence intervals. Ordinal
logistic regression was used to investigate the relationship  between the clinical context and
grading.

Results:

51
rheumatologists completed the study (25 with and 26 without context), from 15
countries (mainly the US, UK, Canada and Italy). A total of 4590 (4080 unique
and 510 repeated) image gradings were obtained. The intra and inter-rater
reliability both without and with the context is presented in Table 1. There
was no significant increase in the overall intra- or inter-rater reliability
with the clinical context. There was a trend towards an increase in intra-rater
reliability for ‘no ulcers’ vs. ‘inactive ulcers’ and ‘no ulcers’ vs.
‘inactive’/’active ulcers’, and a decrease for ‘inactive ulcers’ vs. ‘active
ulcers’. Patient reported discharge (OR= 2.67, 95% c.i. 2.02 to 3.53, P =
<0.001) and pain VAS (OR= 1.02, 95% c.i. 1.01 to 1.03, P = <0.001) were
associated with increased lesion severity, and lesion duration with reduced
severity (OR= 0.81, 95% c.i. 0.76 to 0.86, P = <0.001).

Conclusion:

1.
The overall intra and inter-rater reliability did not significantly improve
with added clinical context.

2.
There was a trend that clinicians may use the clinical context to help classify
lesions as ‘no ulcer’.

3.
Patient reported discharge and pain were associated with increased lesion
severity and duration with reduced severity.

Table 1: Intra- and inter-rater reliability without
and with context.