ACR Meeting Abstracts

ACR Meeting Abstracts

  • Meetings
    • ACR Convergence 2024
    • ACR Convergence 2023
    • 2023 ACR/ARP PRSYM
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • ACR Meetings

Abstract Number: 1222

Large Language Model-driven Sentiment Analysis for Facilitating Fibromyalgia Diagnosis

Vincenzo Venerito and Florenzo Iannone, Rheumatology Unit- University of Bari "Aldo Moro", IT, Bari, Italy

Meeting: ACR Convergence 2024

Keywords: Biomarkers, fibromyalgia, pain

  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
Session Information

Date: Sunday, November 17, 2024

Title: Pain in Rheumatic Disease Including Fibromyalgia Poster

Session Type: Poster Session B

Session Time: 10:30AM-12:30PM

Background/Purpose: Fibromyalgia (FM) is a complex disorder with widespread pain and emotional distress, posing diagnostic challenges. FM patients show altered cognitive and emotional processing, with a preferential allocation of attention to pain-related information. This attentional bias towards pain cues can impair cognitive functions such as inhibitory control, affecting patients’ ability to manage and express their emotions. Sentiment analysis using large language models (LLMs) can provide insights by detecting nuances in pain expression. 

In this proof-of-concept study we investigated whether a LLM-driven sentiment analysis might also catch the nuance of pain expression patterns in FM due to specific lexicon by analyzing linguistic patterns and emotional cues.

Methods: 40 patients with ACR2016-classified FM and 40 non-FM chronic pain controls referred to rheumatology clinics were enrolled. Transcribed responses to questions on pain and sleep were machine translated to English and analyzed by the open-source LLM Mistral-7B-Instruct-v0.2 using prompt engineering targeting FM-associated nuances of pain expression (“prompt-engineered”) or an approach without this targeting (“ablated”) (Figure 1 Panel A). Accuracy, precision, recall, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated using rheumatologist diagnosis as ground truth

Results: The prompt-engineered approach demonstrated accuracy of 0.87, precision of 0.92, recall of 0.84, specificity of 0.82, and AUROC of 0.86 for distinguishing FM (Figure 2). In comparison, the ablated approach had accuracy of 0.76, precision of 0.75, recall of 0.77, specificity of 0.75, and AUROC of 0.76 (Figure 1 Panel B).  The accuracy was superior to the ablated approach (McNemar’s test p< 0.001).

Conclusion: This proof-of-concept study suggests LLM-driven sentiment analysis, especially with prompt engineering, may facilitate FM diagnosis by detecting subtle differences in pain expression. The high accuracy, precision and recall of the prompt-engineered sentiment analysis suggest it could effectively distinguish between patients with and without FM based on subtle differences in their expression of pain symptoms.  Further validation is warranted, particularly inclusion of secondary FM patients.

Supporting image 1

Figure 1. Panel A. Sentiment Analysis Workflow. Panel B. Confusion Matrices. Left matrix: Confusion matrix for prompt-engineered sentiment analysis. Right matrix: Confusion matrix for ablated sentiment analysis. The number of true positives decreased, while false positives increased with ablated sentiment analysis.


Disclosures: V. Venerito: None; F. Iannone: AstraZeneca, 2, GSK, 2, Pfizer, 2, UCB, 2.

To cite this abstract in AMA style:

Venerito V, Iannone F. Large Language Model-driven Sentiment Analysis for Facilitating Fibromyalgia Diagnosis [abstract]. Arthritis Rheumatol. 2024; 76 (suppl 9). https://acrabstracts.org/abstract/large-language-model-driven-sentiment-analysis-for-facilitating-fibromyalgia-diagnosis/. Accessed .
  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print

« Back to ACR Convergence 2024

ACR Meeting Abstracts - https://acrabstracts.org/abstract/large-language-model-driven-sentiment-analysis-for-facilitating-fibromyalgia-diagnosis/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

All abstracts accepted to ACR Convergence are under media embargo once the ACR has notified presenters of their abstract’s acceptance. They may be presented at other meetings or published as manuscripts after this time but should not be discussed in non-scholarly venues or outlets. The following embargo policies are strictly enforced by the ACR.

Accepted abstracts are made available to the public online in advance of the meeting and are published in a special online supplement of our scientific journal, Arthritis & Rheumatology. Information contained in those abstracts may not be released until the abstracts appear online. In an exception to the media embargo, academic institutions, private organizations, and companies with products whose value may be influenced by information contained in an abstract may issue a press release to coincide with the availability of an ACR abstract on the ACR website. However, the ACR continues to require that information that goes beyond that contained in the abstract (e.g., discussion of the abstract done as part of editorial news coverage) is under media embargo until 10:00 AM ET on November 14, 2024. Journalists with access to embargoed information cannot release articles or editorial news coverage before this time. Editorial news coverage is considered original articles/videos developed by employed journalists to report facts, commentary, and subject matter expert quotes in a narrative form using a variety of sources (e.g., research, announcements, press releases, events, etc.).

Violation of this policy may result in the abstract being withdrawn from the meeting and other measures deemed appropriate. Authors are responsible for notifying colleagues, institutions, communications firms, and all other stakeholders related to the development or promotion of the abstract about this policy. If you have questions about the ACR abstract embargo policy, please contact ACR abstracts staff at [email protected].

Wiley

  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences

© Copyright 2025 American College of Rheumatology