Background/Purpose:

Ultrasound is emerging as an alternative test to performing a temporal artery biopsy in the diagnosis of giant cell arteritis (GCA). Little is known of the variability in interpretation of these tests by sonographers and pathologists. We undertook an inter-observer analysis to assess agreement between sonographers in interpreting ultrasound videos, and between pathologists for biopsy images, in patients with suspected GCA.

Methods:

We developed a web exercise with 30 cases randomly sampled from patients with suspected GCA recruited to a large multicentre study comparing ultrasound with biopsy for the diagnosis of GCA. We used 5 practice cases, followed by the 30 unique cases and 6 interspersed repeats, showing ultrasound videos of both temporal arteries and high quality scanned images of biopsies. Trained sonographers and pathologists from the study were asked to assess the compatibility of the videos and images with a diagnosis of GCA and indicate how confident they were of the diagnosis. Inter-observer agreement between sonographers and between pathologists was evaluated using two-way random effects analysis of variance to estimate the intra-class correlation coefficient (ICC) for agreement. Intra-observer reproducibility was evaluated using kappa statistics for the 6 repeated cases.

Results:

All 12 sonographers agreed unanimously on 10/30 cases; 4 as GCA and 6 as not GCA. In 5 cases at least 3 sonographers differed from the majority. All 14 pathologists agreed unanimously on 11 cases; 6 as GCA and 5 as not GCA. In 5 cases at least 3 differed from the majority and in 1 case the pathologists were evenly divided between GCA and not GCA. Overall agreement was similar between the two groups: the ICC for sonographers 0.61 (95% CI 0.48, 0.75) and for pathologists 0.62 (95% CI 0.49, 0.76). After allowing for confidence in the interpretation of the videos and images, inter-observer agreement between sonographers (ICC 0.58, 95% CI 0.44, 0.72) was lower than between pathologists (ICC 0.72, 95% CI 0.60, 0.83). Evidence of giant cells was indicated by the original reporting pathologist in 8/30 cases; in 6/8, the pathologists unanimously judged the biopsy images to be consistent with GCA. For intra-observer reproducibility of the repeated cases the sonographers (raw agreement 86%, kappa 0.69) performed less well than the pathologists (raw agreement 92%, kappa 0.83). 

Conclusion:

We have shown that the level of agreement between sonographers is similar to that between pathologists for assessing the compatibility of videos or images with a diagnosis of GCA. Pathologists performed better after allowing for certainty in interpretation and for intra-observer reproducibility. However, the level of agreement was not as high as might be expected and suggests scope for improved training for interpreting temporal artery ultrasound and further investigation of how histological abnormalities other than giant cells are interpreted by pathologists. Interpreting results to support or overturn a diagnosis of suspected GCA should be undertaken in the light of these findings.

« Back to 2015 ACR/ARHP Annual Meeting

ACR Meeting Abstracts - https://acrabstracts.org/abstract/inter-rater-analysis-of-ultrasound-and-histological-findings-in-patients-with-suspected-giant-cell-arteritis/