ACR Meeting Abstracts

ACR Meeting Abstracts

  • Home
  • Meetings Archive
    • ACR Convergence 2022
    • ACR Convergence 2021
    • ACR Convergence 2020
    • 2020 ACR/ARP PRSYM
    • 2019 ACR/ARP Annual Meeting
    • 2018 ACR/ARHP Annual Meeting
    • 2017-2009 Meetings
    • Download Abstracts
  • Keyword Index
  • Advanced Search
  • Your Favorites
    • Favorites
    • Login
    • View and print all favorites
    • Clear all your favorites
  • Meeting Resource Center

Abstract Number: 94

Validity Evidence for an Objective Structured Clinical Examination Station to Assess Knee Arthrocentesis Skill

Tawnie Braaten1, Andrea Barker2, J. Peter Beck3 and Michael J. Battistone2, 1University of Utah, Salt Lake City, UT, 2Veterans Affairs Salt Lake City Health Care System and University of Utah School of Medicine, Salt Lake City, UT, 3Orthopaedics, Salt Lake City VA Medical Center and University of Utah, Salt Lake City, UT

Meeting: 2017 ACR/ARHP Annual Meeting

Date of first publication: September 18, 2017

Keywords: Assessment, Education, educational innovation, educational research and educator, medical

  • Tweet
  • Email
  • Print
Session Information

Date: Sunday, November 5, 2017

Session Title: Education Poster

Session Type: ACR Poster Session A

Session Time: 9:00AM-11:00AM

Background/Purpose:

We developed an objective structured clinical examination (OSCE) station to guide preceptors’ observations of trainees’ performance of simulated knee arthrocentesis and to organize feedback in preparation for performing these procedures in patient care. The aim of this project was to examine the evidence for validity of this OSCE.

Methods:

Content:

An orthopedic surgeon, a rheumatologist, and a primary care provider with expertise in musculoskeletal (MSK) medicine developed a checklist to guide rater observations in evaluating arthrocentesis technique. Content was proposed by faculty, supplemented by literature review, and finalized through consensus.

 

Response Process

A multi-disciplinary cohort of 71 learners (53 postgraduate trainees, 12 physician assistant students, 5 advanced practice nursing students, and one university undergraduate student) participated in the OSCEs in 2016-2017. To promote accuracy of the simulated patient (SP) responses to assessment prompts, one faculty member served as the SP and another as rater; ratings were recorded in real time.

 

Internal Structure

Two faculty members independently rated a portion of the cases. Percent agreement was calculated and Cohen’s kappa corrected for chance agreement on binary outcomes.

 

Relations to other variables

Relationship to self-assessment of confidence and competence to perform knee arthrocentesis was explored by written surveys utilizing a 5-point Likert scale. Response scores were compared with OSCE scores through Pearson’s correlation coefficient.

Results:

Checklists were developed for knee arthrocentesis (19 items). The checklist was scored by assigning one point for each error observed, in order to capture multiple errors; thus a score of 0 indicated no errors were noted. Mean score was 3.7 (range = 0-8; s.d. = 1.9). Frequency distribution of scores is reported in the Figure:

 

Inter-rater agreement was near perfect at 93% (k = 0.8), as shown in the Table:

 

 

Rater 1

 

No Error Noted

Error Noted

Total

Rater 2

No Error Noted

116

5

121

Error Noted

6

31

37

Total

122

26

158

Observed Agreement = (116 + 31)/158 = 0.93

Chance Agreement = 0.64

Cohen’s kappa = 0.8

Pearson’s coefficient indicated no correlation between self-assessment and OSCE performance (-0.002).

Conclusion:  

Validity evidence supports use of this OSCE in educational programs preparing learners for clinical settings where MSK procedures may be performed. Evidence for validity includes systematic development of content, response process control, and demonstration of acceptable interrater agreement. Lack of correlation with self-assessments suggests that the OSCE measures a construct different than self-perceived ability.


Disclosure: T. Braaten, None; A. Barker, None; J. P. Beck, None; M. J. Battistone, None.

To cite this abstract in AMA style:

Braaten T, Barker A, Beck JP, Battistone MJ. Validity Evidence for an Objective Structured Clinical Examination Station to Assess Knee Arthrocentesis Skill [abstract]. Arthritis Rheumatol. 2017; 69 (suppl 10). https://acrabstracts.org/abstract/validity-evidence-for-an-objective-structured-clinical-examination-station-to-assess-knee-arthrocentesis-skill/. Accessed January 28, 2023.
  • Tweet
  • Email
  • Print

« Back to 2017 ACR/ARHP Annual Meeting

ACR Meeting Abstracts - https://acrabstracts.org/abstract/validity-evidence-for-an-objective-structured-clinical-examination-station-to-assess-knee-arthrocentesis-skill/

Advanced Search

Your Favorites

You can save and print a list of your favorite abstracts during your browser session by clicking the “Favorite” button at the bottom of any abstract. View your favorites »

ACR Pediatric Rheumatology Symposium 2020

© COPYRIGHT 2023 AMERICAN COLLEGE OF RHEUMATOLOGY

Wiley

  • Home
  • Meetings Archive
  • Advanced Search
  • Meeting Resource Center
  • Online Journal
  • Privacy Policy
  • Permissions Policies
  • Cookie Preferences