Background/Purpose: Fibromyalgia (FM) is a complex disorder with widespread pain and emotional distress, posing diagnostic challenges. FM patients show altered cognitive and emotional processing, with a preferential allocation of attention to pain-related information. This attentional bias towards pain cues can impair cognitive functions such as inhibitory control, affecting patients’ ability to manage and express their emotions. Sentiment analysis using large language models (LLMs) can provide insights by detecting nuances in pain expression. 

In this proof-of-concept study we investigated whether a LLM-driven sentiment analysis might also catch the nuance of pain expression patterns in FM due to specific lexicon by analyzing linguistic patterns and emotional cues.

Methods: 40 patients with ACR2016-classified FM and 40 non-FM chronic pain controls referred to rheumatology clinics were enrolled. Transcribed responses to questions on pain and sleep were machine translated to English and analyzed by the open-source LLM Mistral-7B-Instruct-v0.2 using prompt engineering targeting FM-associated nuances of pain expression (“prompt-engineered”) or an approach without this targeting (“ablated”) (Figure 1 Panel A). Accuracy, precision, recall, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated using rheumatologist diagnosis as ground truth

Results: The prompt-engineered approach demonstrated accuracy of 0.87, precision of 0.92, recall of 0.84, specificity of 0.82, and AUROC of 0.86 for distinguishing FM (Figure 2). In comparison, the ablated approach had accuracy of 0.76, precision of 0.75, recall of 0.77, specificity of 0.75, and AUROC of 0.76 (Figure 1 Panel B).  The accuracy was superior to the ablated approach (McNemar’s test p< 0.001).

Conclusion: This proof-of-concept study suggests LLM-driven sentiment analysis, especially with prompt engineering, may facilitate FM diagnosis by detecting subtle differences in pain expression. The high accuracy, precision and recall of the prompt-engineered sentiment analysis suggest it could effectively distinguish between patients with and without FM based on subtle differences in their expression of pain symptoms.  Further validation is warranted, particularly inclusion of secondary FM patients.

« Back to ACR Convergence 2024

ACR Meeting Abstracts - https://acrabstracts.org/abstract/large-language-model-driven-sentiment-analysis-for-facilitating-fibromyalgia-diagnosis/