Background/Purpose: Many in-person research studies, such as ours examining cognitive impairment (CI) in SLE, were paused for safety reasons during the COVID-19 pandemic. To restart the study, cognitive testing originally administered in-person moved to a virtual platform. This required alterations to some tests. This analysis aimed to determine if the administration method (in-person vs virtual) of ACR recommended cognitive tests (ACR-NB) affected participant performance and classification. Participants were also asked if they preferred in-person or virtual assessments.

Methods: Data from our multi-visit, longitudinal, SLE CI study included demographic, clinical and psychiatric characteristics and the modified ACR-NB. Data collection was repeated at 0, 6, 12 and 24 months. The ACR-NB was originally administered in-person (IP), then switched to virtual (V) during the pandemic. Two of the 19 ACR-NB tests could not be administered virtually and three required modification from written to verbal responses for the virtual visits (Table 1). Participant performance across visits was classified as persistently cognitively impaired, fluctuating CI, and non-CI.
Kruskal-Wallis, Mann-Whitney-U or chi-square tests examined group (in-person vs virtual) differences for:
1. Demographic, clinical and psychiatric characteristics (all visits)
2. Cognitive performance (ACR-NB) at:
a. All visits
b. Non-CI group visits only
c. Within person comparison for participants with both an in-person and virtual visit
A retrospective preferences questionnaire was given to participants who completed the ACR-NB both in-person and virtually.

Results: We analysed 328 SLE participants who had 801 visits (696 IP, 105 V). Demographic, clinical and psychiatric characteristics were comparable except the virtual visits had a higher anxiety score (p< 0.001). 10/17 cognitive tests were significantly different for the all visit comparison. For the non-CI visits (394 visits, 346 IP versus 48 V) and within comparisons (n=71), 7/17 cognitive tests were significantly different between virtual and in-person. Across all three comparisons, six tests were consistently statistically significant (Table 2). 12/71 (17%) of participants had a change in CI status due to the seven tests that were significantly different (four now classified with CI and eight no longer classified with CI). 61% of participants completing the preference questionnaire preferred virtual administration.

Conclusion: Of the 19 tests in the ACR-NB, we identified one or more problems with eight (42%) tests as we moved from in-person to virtual administration including: tests that cannot be administered virtually (n=2); tests that required modification (n=3), and tests (n=6) that showed significant and consistent performance differences according to mode of administration. As use of virtual cognitive testing will likely increase due to advances in technology and participant preference, these issues need to be addressed – potentially by validating a virtual version of the ACR-NB. Until then, caution must be taken when directly comparing virtual to in-person testing results. Future studies should avoid using a mixed administration approach.

ACR Meeting Abstracts - https://acrabstracts.org/abstract/are-virtual-cognitive-assessments-comparable-to-in-person-assessments-in-a-systemic-lupus-erythematosus-cohort/