ACR Meeting Abstracts

ACR Meeting Abstracts

  • Meetings
    • ACR Convergence 2024
    • ACR Convergence 2023
    • 2023 ACR/ARP PRSYM
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • ACR Meetings

Abstract Number: 1290

Examining the Quality of Surveys Used in Rheumatology Health Professions Education Research

Madeline O’Sullivan1, Andrea Barker2, Naomi Schlesinger1 and Michael Battistone2, 1University of Utah, Salt Lake City, UT, 2Salt Lake City VA, Salt Lake City, UT

Meeting: ACR Convergence 2024

Keywords: education, medical, Surveys

  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print
Session Information

Date: Sunday, November 17, 2024

Title: Professional Education Poster

Session Type: Poster Session B

Session Time: 10:30AM-12:30PM

Background/Purpose: Health professions education (HPE) is an evolving discipline that frequently utilizes surveys to collect data about abstract concepts. A study of the methodologic rigor of surveys published in leading HPE journals raised concerns about their quality (Artino, 2018). Surveys that violate best practices in design can result in measurement errors and problems with reliability and validity.

A growing community of HPE researchers is recognized within ACR, and HPE scholarship is increasingly published in rheumatology journals. The prevalence and quality of the surveys used have not been examined. This gap is important to education scholars who conduct research relevant to the future workforce, sponsors, and funding agencies that support this work, and reviewers who appraise the scientific merit of a report.

This study aimed to determine the numbers of HPE studies that used surveys published in leading rheumatology journals, assess their quality using a previously described rubric, and examine the frequency of response rate reporting.

Methods: Two authors (MO, MJB) searched articles published in leading rheumatology journals over a 5-year period from 2019 through 2023 for those that used at least one survey tool and included health professionals as respondents. Titles and abstracts were independently screened for eligibility, and inclusion decisions were made in consensus between MO and MJB. Quality was assessed using a coding rubric which evaluated the reliability and validity of the reported evidence and examined the methodologic rigor of specific survey items.

All authors independently coded four papers and then met to compare ratings and resolve differences; three authors (MO, AMB, MJB) divided the remaining publications so that each article was independently coded twice.

Two authors (MO, MJB) also reviewed ACR abstracts presented over the same period to identify those that covered an HPE topic and utilized surveys.

Results: Twenty articles met inclusion criteria. Eleven (55%) included a complete copy of the fully formatted survey for review. Six (30%) were adapted from previously published surveys.  Response rates were reported in 13 (65%) of studies, ranging from 5% to 100%. Reliability and validity evidence were reported for 11 (55%) and 1 (5%) of the surveys, respectively (Table 1). Nine of the 11 (81%) surveys with a fully formatted copy violated at least one best practice for design, with the most common violations being agreement-response and multi-barreled items (Table 2).

Among 141 presentations at the ACR annual meetings education poster and concurrent abstract sessions from 2019 to 2023, 84 (60%) included surveys (Table 3). The prevalence of survey use in these abstracts was stable, ranging from 55-63%.

Conclusion: Surveys are prominent in HPE scholarship presented at ACR education abstract sessions. The majority of HPE research manuscripts that include surveys published in leading rheumatology journals failed to report validity and reliability evidence, and many violated established best practices in survey design. Improving methodologic rigor in designing, using, and reporting surveys in rheumatology education research will strengthen the data we collect for the questions we seek to answer.

Supporting image 1

Table 1. Frequency of validity and reliability evidence reporting by authors of survey-based research articles published in leading rheumatology journals from 2019 to 2023

Supporting image 2

Table 2. Percentage of surveys with fully formatted copy available for review (n=11) that violated the five survey design tenets

Supporting image 3

Table 3. HPE abstracts presented at ACR utilizing surveys from 2019 to 2023


Disclosures: M. O’Sullivan: None; A. Barker: None; N. Schlesinger: arthrosi, 1, horizon, 1, Novartis, 1, olatec, 1, 2, ptotalix, 1, 2, shanton, 1, sobi, 1; M. Battistone: None.

To cite this abstract in AMA style:

O’Sullivan M, Barker A, Schlesinger N, Battistone M. Examining the Quality of Surveys Used in Rheumatology Health Professions Education Research [abstract]. Arthritis Rheumatol. 2024; 76 (suppl 9). https://acrabstracts.org/abstract/examining-the-quality-of-surveys-used-in-rheumatology-health-professions-education-research/. Accessed .
  • Tweet
  • Click to email a link to a friend (Opens in new window) Email
  • Click to print (Opens in new window) Print

« Back to ACR Convergence 2024

ACR Meeting Abstracts - https://acrabstracts.org/abstract/examining-the-quality-of-surveys-used-in-rheumatology-health-professions-education-research/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

All abstracts accepted to ACR Convergence are under media embargo once the ACR has notified presenters of their abstract’s acceptance. They may be presented at other meetings or published as manuscripts after this time but should not be discussed in non-scholarly venues or outlets. The following embargo policies are strictly enforced by the ACR.

Accepted abstracts are made available to the public online in advance of the meeting and are published in a special online supplement of our scientific journal, Arthritis & Rheumatology. Information contained in those abstracts may not be released until the abstracts appear online. In an exception to the media embargo, academic institutions, private organizations, and companies with products whose value may be influenced by information contained in an abstract may issue a press release to coincide with the availability of an ACR abstract on the ACR website. However, the ACR continues to require that information that goes beyond that contained in the abstract (e.g., discussion of the abstract done as part of editorial news coverage) is under media embargo until 10:00 AM ET on November 14, 2024. Journalists with access to embargoed information cannot release articles or editorial news coverage before this time. Editorial news coverage is considered original articles/videos developed by employed journalists to report facts, commentary, and subject matter expert quotes in a narrative form using a variety of sources (e.g., research, announcements, press releases, events, etc.).

Violation of this policy may result in the abstract being withdrawn from the meeting and other measures deemed appropriate. Authors are responsible for notifying colleagues, institutions, communications firms, and all other stakeholders related to the development or promotion of the abstract about this policy. If you have questions about the ACR abstract embargo policy, please contact ACR abstracts staff at [email protected].

Wiley

  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences

© Copyright 2025 American College of Rheumatology