Background/Purpose: In the classification criteria of Sjögren’s syndrome (SS), involvement of the salivary glands is currently assessed by sialography, scintigraphy, sialometry and histopathology. Recent discussions have focused on the diagnostic accuracy of ultrasound (US). The purpose of this study was to determine inter- and intra-observer  reliability of US of major salivary glands in patients clinically suspected with SS.

Methods: Eighty consecutive patients clinically suspected with SS were subjected to US as part of their diagnostic work-up. The following US variables of the parotid and submandibular salivary glands were assessed: echogenicity, parenchymal homogeneity, presence of hypoechogenic areas, hyperechogenic reflections and clearness of posterior glandular border (scoring range 0-48)¹. The cut-off point to define positive or negative US for SS was set at 15 ²  and 17 ¹. Images were scored independently by three blinded observers in 2 sessions with a 2-week interval. Intra- and inter-observer reliability was calculated using Cohen’s Kappa and Fleiss’ Kappa, respectively, in combination with the percentage of absolute agreement for nominal variables and intraclass correlation coefficients (ICC) for continuous variables.

Results: Intra-observer reliability of the US total score was excellent, with ICCs ranging from 0.89 to 0.96 for the three observers. Inter-observer reliability was good to excellent, with ICCs of 0.84 and 0.76 for the US total score in sessions one and two, respectively. Kappa ranged from 0.60 to 0.83 and percentage of agreement from 80 to 92 depending on the cut-offs applied. Hypoechogenic areas and homogeneity of the parotid glands showed the highest inter-observer reliability with median ICCs of 0.74 and 0.71, respectively. Median kappa for echogenicity was low (0.22). The differences between the three observers were larger for higher US total scores (Figure). Our results suggest that observers may consistently identify in which patients US of the major salivary glands is positive or negative for SS, but scoring the severity of the US findings is more inconsistent.

Conclusion: US of the major salivary glands is reliable in diagnosing SS. There might be, however, some discrepancies between observers in assessing the severity of US findings making it more difficult to detect ‘true’ changes over time. Thus, when monitoring the progression of SS or treatment efficacy, it is advised that each particular patient is scored by the same ultrasonographer at every time point. References

1.    Hocevar et al. Rheumatology (Oxford) 2005;44 :768-772

2.    Zhang et al. Rheumatology (Oxford). 2015;54:1680-7 Figure: Systematic differences in US total score. For each patient, the mean of the 6 observations (3 observers,  2 sessions) and the difference of these 6 observations with the mean were calculated and plotted against each other. The intermittent vertical lines indicate the cut-off points applied.