Session Type: ACR Poster Session B
Session Time: 9:00AM-11:00AM
Evidence for 2 Objective Structured Clinical Examination Stations to Assess
Core Examination Skills of the Shoulder and Knee
multi-disciplinary group developed two objective structured clinical
examination (OSCE) stations to train and assess trainee’s skills in the
evaluation and management of shoulder pain and knee pain. Our objective was to
examine the validity of these two OSCEs.
orthopaedists, two rheumatologists, and a primary care provider developed checklists
of physical exam maneuvers and criteria for guiding rater observations. Content
was proposed by faculty, supplemented by literature review, and finalized through
a modified Delphi process. Simulated cases representing common causes of
shoulder pain and knee pain were constructed.
multi-disciplinary cohort of 69 trainees participated in the OSCEs in 2014-15. To
promote accuracy of the simulated patient (SP) responses to assessment prompts,
one faculty member served as SP and another as rater; ratings were recorded in
faculty members independently rated a portion of the cases. Percent agreement
was calculated and Cohen’s kappa corrected for chance agreement on binary
to self-assessment of ability to evaluate shoulder pain and knee pain was
explored by written surveys utilizing a 5-point Likert scale. Responses were stratified
into 3 categories – low, medium, and high – and compared with similarly
stratified OSCE scores.
were developed for the shoulder (21 items) and knee (26 items); scoring rubrics
converted each checklist to a 5-point scale. Using the examination approach in
the checklists, trainees correctly identified rotator cuff pathology 61/69 (88%)
and meniscal disease 62/69(89%) of the time. Inter-rater agreement was 89% for
the knee (k = 0.55) and 96% for the shoulder (k = 0.51). Relationship of
stratified self-assessment and OSCE scores is shown in Table 1; Pearson’s
coefficient indicated no correlation for either shoulder (0.02) or knee (-0.07).
two station OSCE is responsive and consistency in test performance across a
range of trainees over time, with good inter-rater reliability. Lack of
correlation with self-assessment suggests that these OSCEs measure a construct
that is different than learners’ self-confidence to support the continuing use
of these two shoulder and knee OSCE stations in a structured educational
program. The next steps will search for additional validity evidence of these
To cite this abstract in AMA style:Battistone MJ, Barker AM, Beck JP, Tashjian RZ, Cannon GW. Validity Evidence for Two Objective Structured Clinical Examination Stations to Assess Core Examination Skills of the Shoulder and Knee [abstract]. Arthritis Rheumatol. 2015; 67 (suppl 10). https://acrabstracts.org/abstract/validity-evidence-for-two-objective-structured-clinical-examination-stations-to-assess-core-examination-skills-of-the-shoulder-and-knee/. Accessed January 18, 2020.
« Back to 2015 ACR/ARHP Annual Meeting
ACR Meeting Abstracts - https://acrabstracts.org/abstract/validity-evidence-for-two-objective-structured-clinical-examination-stations-to-assess-core-examination-skills-of-the-shoulder-and-knee/