Background/Purpose: Holistic review is an approach to candidate selection that emphasizes alignment of candidate attributes to a program’s mission. Identifying character strengths in personal statements and letters of recommendation within fellowship applications is a vital but time-consuming and subjective task. The increasing volume and competitiveness of rheumatology fellowship applications raises the stakes even further. We describe a natural language processing (NLP) approach to identify VIA (Values in Action) Character Strengths in fellowship application personal statements and letters of recommendation, providing a data-driven method to support holistic applicant review based on the principles of positive psychology.

Methods: This feasibility study was conducted in three steps: (1) representation of character strengths, (2) computer assignment of character strengths, and (3) comparison to human-based approaches. First, two physicians first represented the 24 VIA character strengths using documentation from the VIA Institute and peer-reviewed literature. For each character strength, five phrases were developed that aligned with the University of Iowa rheumatology fellowship program’s mission statement.Secondly, an algorithm using NLP was devised using the python computer language that employed sentence-BERT (SBERT) embeddings. The algorithm analyzes semantic similarity of sentences within the personal statements and letters of recommendation to each of 24 VIA Character Strengths. For each candidate, a score between 0 and 1 was assigned for the presence of each strength. This was then represented as a heat map for easy visualization of each candidate’s character strengths.Lastly, this was repeated three times to ensure consistency of computer-generated results. Two physicians independently reviewed the personal statements and letters of recommendation for character strengths. Results of the human- and computer-based approaches were then compared for consistency and for time to complete the task.

Results: The algorithm successfully identified character strengths across 9 personal statements and 27 letters of recommendation from a sample of 9 prior applications, demonstrating individual differences in the representation of character strengths within candidate applications.There was high concordance between character strengths assigned by human reviewers and those identified by the algorithm (kappa=0.89). The strengths with highest concordance between the algorithm and human reviewers were teamwork and leadership, while the lowest were spirituality and forgiveness.Output and performance of the model was consistent and reduced the time to identify strengths from 240 minutes to 35 seconds.

Conclusion: This study presents a novel NLP-based algorithm for analyzing personal statements, offering a scalable and customizable approach for rheumatology program directors and other leaders to rapidly evaluate character strengths. By integrating machine learning into holistic review based on a positive psychology framework, this method provides structured insights into applicants’ personal attributes, complementing holistic selection processes in a rapid and reproducible manner.

This feasibility study consisted of three steps: (1) Representation and calibration of character strengths with the program mission statement, (2) Development of the NLP-algorithm, and (3) Comparison of NLP & human approaches

This heatmap illustrates the distribution of character strengths among the documents for each of the fellowship candidates. Such a heatmap can be used to rapidly appreciate representations of character strengths within candidate applications.

« Back to ACR Convergence 2025

ACR Meeting Abstracts - https://acrabstracts.org/abstract/using-natural-language-processing-to-identify-character-strengths-in-rheumatology-fellowship-candidate-applications-a-feasibility-study/