ACR Meeting Abstracts

ACR Meeting Abstracts

  • Meetings
    • ACR Convergence 2024
    • ACR Convergence 2023
    • 2023 ACR/ARP PRSYM
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • ACR Meetings

Abstract Number: 1276

Using Classification and Regression Tree Analysis to Assess the Construct Validity of the Automated Neuropsychological Assessment Metrics in the Assessment of Cognitive Impairment in SLE Compared to the ACR Neuropsychological Battery

Kimberley Yuen1, Dorcas Beaton2, Kathleen Bingham3, Jiandong Su4, Mahta Kakvan4, Juan Pablo Diaz-Martinez4, Carmela Tartaglia5, Lesley Ruttan6, Joan Wither4, Nicole Anderson4, Dennisse Bonilla4, May Choi7, Marvin Fritzler8, Patricia Katz9, Robin Green6 and Zahi Touma10, 1Queen's University School of Medicine; University of Toronto Lupus Clinic, Centre for Prognosis Studies in the Rheumatic Diseases, Toronto Western Hospital, University Health Network, Toronto, Canada, 2Institute for Work & Health, Toronto, ON, Canada, 3Department of Psychiatry, University of Toronto, Toronto, ON, Canada, 4University of Toronto Lupus Clinic, Centre for Prognosis Studies in Rheumatic Diseases, Toronto Western Hospital, University Health Network, Toronto, ON, Canada, 5University of Toronto, Krembil Neurosciences Centre, Toronto, ON, Canada, 6Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada, 7Brigham and Women's Hospital | Cumming School of Medicine, University of Calgary, Calgary, AB, Canada, 8Department of Medicine, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada, 9University of California, San Francisco, Novato, CA, 10University of Toronto Lupus Clinic, Centre for Prognosis Studies in Rheumatic Diseases, Toronto Western Hospital, University Health Network; Krembil Research Institute, Toronto, ON, Canada

Meeting: ACR Convergence 2020

Keywords: Cognitive dysfunction, Neuropsychological Assessment, Systemic lupus erythematosus (SLE)

  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
Session Information

Date: Sunday, November 8, 2020

Title: SLE – Diagnosis, Manifestations, & Outcomes Poster II: Comorbidities

Session Type: Poster Session C

Session Time: 9:00AM-11:00AM

Background/Purpose: Cognitive impairment (CI) is common in systemic lupus erythematosus (SLE) patients, however there is no standard screening tool available. The American College of Rheumatology Neuropsychological Battery (ACR-NB) is the gold standard diagnostic tool but requires trained personnel and 1-hour to complete. The Automated Neuropsychological Assessment Metrics (ANAM) is a promising screening tool as it is a shorter, self-administered battery. We previously developed a composite index for interpreting ANAM, and in this study we aimed to provide further evidence on the construct validity of the ANAM as a screening tool for detecting CI in SLE compared to ACR-NB. Objectives: 1) to determine the best subtests and score types of ANAM to predict CI using Classification and Regression Tree (CART) analysis, 2) provide a new approach for interpreting ANAM results, and 3) compare the performance of ANAM composite index to the best CART model.

Methods: 300 consecutive consenting adult SLE patients attending a single center between 2016 and 2019 completed the ANAM and ACR-NB on the same day. Patients were classified CI or non-CI based on ACR-NB. CI was defined by a z-score of ≤-1.5 in ≥ 2 domains; non-CI if no domain had a z-score of ≤-1.5. ANAM has 15 subtests with each test comprising different score types (percentage of correct responses [PCT], mean reaction time [MR], throughput [TP], and coefficient of variation [CV]). We built 6 models including all subtests, but each model included a different set of score types. Each model was run through CART analyses with k-fold cross validation. The data was split into k-subsets (k=10). Each subset randomly selected 90% of patients as training data and 10% as testing data, repeated 3 times. A decision tree was generated for each model separating patients into CI or non-CI based on ANAM subtests, score types, and thresholds that best predict CI. We hypothesized ANAM would identify CI with good Area Under the Receiver Operating Characteristics (ROC) curves (AUC = 0.71-0.80) and a sensitivity ≥ 80%. The performance of the best CART model was compared to ANAM composite index using ROC analysis and kappa statistic. Composite Index: 3.88-0.05*Code Substitution DelayedPCT – 8.4*Spatial ProcessingCV + 2.44*Code Substitution LearningMR + 9.87*Tapping LeftMR.

Results: 145 patients were defined as CI and 90 as non-CI based on ACR-NB. The best model included 6 ANAM subtests and PCT, MR, CV score types (Figure 1), with 89% sensitivity, 69% specificity, AUC of 81% (95% confidence interval: 75-86%), 81% accuracy, 82% positive predictive value, and 79% negative predictive value compared to ACR-NB. The most important subtests, score types, and thresholds were displayed in a decision tree (Figure 1). Concordance between the best CART model and ANAM composite index was 74%, κ = 0.48 (95% confidence interval: 0.37-0.59) (Figure 2).

Conclusion: Our findings extend the evidence on the construct validity of the ANAM as a screening tool for CI in SLE patients. Good concordance was found between the ANAM composite index and best CART model. Six most important subtests and scores for detecting CI from the ANAM were identified. A decision tree was generated to simplify the interpretation of ANAM and enhance clinical utility.

CI: cognitive impairment; PCT: percentage correct responses; MR: mean reaction time; CV: coefficient of variation of mean reaction time; TP: throughput For most of the ANAM tests, there are 4 score types provided. PCT represents accuracy. MR represents the average reaction time. TP is a measure of cognitive efficiency and is derived as the number of correct responses per minute. CV is an index of the patient’s consistency of response speed within a given timed subtest. It is a derived score (standard deviation of MR divided by MR). Better cognitive performance is indicated by higher PCT and TP scores, and lower MR and CV scores. Code substitution delayed measures delayed memory and requires the participant to compare a symbol-digit pair with previously defined and memorized symbol-digit pairs. The participant presses designated buttons to indicate whether the pair was correct or incorrect based on their memory. Tapping left hand and tapping right hand measure motor skill and reaction time. The participant is required to press the spacebar with the designated hand as many times as possible in a single trial. Simple reaction time repeated measures reaction time and requires the participant to respond as quickly as possible to a target stimulus. The “repeated” session is administered later in the battery and is also designed to measure fatigue. Logical relations measures abstract reasoning and verbal fluency whereby the participant evaluates the truth of statements and describes the order of two symbols displayed on the screen. Spatial processing measures visual spatial skills whereby the participant compares two histograms displayed side-by-side in different orientations.

CART: Classification and Regression Tree; AUC: Area under the curve; PCT: Percentage of correct responses; MR: Mean reaction time; CV: Coefficient of variation The ROC curve of the best model (PCT, MR, CV) from CART analysis reported an AUC of 81% compared to ACR-NB. The composite index formula (which was derived using logistic regression from our previous study) based on the model with PCT, MR, CV was applied to the current dataset and reported an AUC of 76% compared to ACR-NB.


Disclosure: K. Yuen, None; D. Beaton, None; K. Bingham, None; J. Su, None; M. Kakvan, None; J. Diaz-Martinez, None; C. Tartaglia, None; L. Ruttan, None; J. Wither, None; N. Anderson, None; D. Bonilla, None; M. Choi, None; M. Fritzler, Inova Diagnostics Inc, 5, 8, Werfen International, 5, 8; P. Katz, None; R. Green, None; Z. Touma, None.

To cite this abstract in AMA style:

Yuen K, Beaton D, Bingham K, Su J, Kakvan M, Diaz-Martinez J, Tartaglia C, Ruttan L, Wither J, Anderson N, Bonilla D, Choi M, Fritzler M, Katz P, Green R, Touma Z. Using Classification and Regression Tree Analysis to Assess the Construct Validity of the Automated Neuropsychological Assessment Metrics in the Assessment of Cognitive Impairment in SLE Compared to the ACR Neuropsychological Battery [abstract]. Arthritis Rheumatol. 2020; 72 (suppl 10). https://acrabstracts.org/abstract/using-classification-and-regression-tree-analysis-to-assess-the-construct-validity-of-the-automated-neuropsychological-assessment-metrics-in-the-assessment-of-cognitive-impairment-in-sle-compared-to-t/. Accessed .
  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print

« Back to ACR Convergence 2020

ACR Meeting Abstracts - https://acrabstracts.org/abstract/using-classification-and-regression-tree-analysis-to-assess-the-construct-validity-of-the-automated-neuropsychological-assessment-metrics-in-the-assessment-of-cognitive-impairment-in-sle-compared-to-t/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

All abstracts accepted to ACR Convergence are under media embargo once the ACR has notified presenters of their abstract’s acceptance. They may be presented at other meetings or published as manuscripts after this time but should not be discussed in non-scholarly venues or outlets. The following embargo policies are strictly enforced by the ACR.

Accepted abstracts are made available to the public online in advance of the meeting and are published in a special online supplement of our scientific journal, Arthritis & Rheumatology. Information contained in those abstracts may not be released until the abstracts appear online. In an exception to the media embargo, academic institutions, private organizations, and companies with products whose value may be influenced by information contained in an abstract may issue a press release to coincide with the availability of an ACR abstract on the ACR website. However, the ACR continues to require that information that goes beyond that contained in the abstract (e.g., discussion of the abstract done as part of editorial news coverage) is under media embargo until 10:00 AM ET on November 14, 2024. Journalists with access to embargoed information cannot release articles or editorial news coverage before this time. Editorial news coverage is considered original articles/videos developed by employed journalists to report facts, commentary, and subject matter expert quotes in a narrative form using a variety of sources (e.g., research, announcements, press releases, events, etc.).

Violation of this policy may result in the abstract being withdrawn from the meeting and other measures deemed appropriate. Authors are responsible for notifying colleagues, institutions, communications firms, and all other stakeholders related to the development or promotion of the abstract about this policy. If you have questions about the ACR abstract embargo policy, please contact ACR abstracts staff at [email protected].

Wiley

  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences

© Copyright 2025 American College of Rheumatology