Background/Purpose
We developed and conducted an OSCE to assess clinical skills of trainees in rheumatology (TRs) and determine its performance at two consecutive annual evaluations of the National Board of Rheumatology (NCR) certification.
Methods
Thirty-two (in 2013) and 38 (in 2014) TRs, underwent an OSCE and a 300-questions examination (MCQ). MCQ was annually developed by faculty and experienced test questions writers NCRS–certified members, who submitted questions based on pre-specified content areas. Each question was reviewed by a committee of 4 NCR members.
OSCE circuits were developed over a 10-month period by a trained NCR committee. At first, NCR members selected and designed stations using public core skills which included history-taking, physical examination, problem solving, studies interpretation, intra-articular injection (using a model) and capillaroscopy test; then, an expert consensus panel of rheumatologists validated each station (≥80% agreement); appropriated consented patients were selected and trained, as were examiners and each one was assigned to a particular station; finally, a pilot OSCE was performed by 3 certified rheumatologists who served as the “gold standard” control participants. Feedback was obtained. Final circuits consisted of 12 (in 2013) and 15 (in 2014) 8-minutes-stations, respectively, with 4 (2013) and 5 (2014) additional rest stations. Stations were scored by the same examiner in a previously validated check-list.
A composite OSCE score was obtained from each participant. Inter-station correlation was calculated using Pearson´s correlation coefficient. Concurrent validity was established by correlating MCQ scores and composite OSCE scores within each TR (Pearson´s correlation coefficient), by comparing OSCE scores between TRs and certified rheumatologists (Student t test) and by comparing distribution of TRs with MCQ pass scores among TRs with/without OSCE pass score (Wilcoxon rank sum).
Results
In 2013, mean (±SD) OSCE score in all the participants was 7.1(±0.6) and none received a failing score, meanwhile mean MCQ was 6.5(±0.6) and 7 TRs (21.9%) received a failing score (<6). In 2014, mean (±SD) OSCE score was 6.7(±0.6) and 3 TRs (7.9%) received a failing score (<6), 2 of whom also received a failing score of the MCQ portion of the examination. Mean MCQ was 6.4(±0.5) and 7 TRs (18.5%) received a failing score, 2 of whom also received a failing score in the OSCE. Thirty TRs (78.9%) received a pass score at both MCQ and OSCE.
In 2013, there was a significant correlation between MCQ score and composite OSCE score (r=0.44, p=0.006) meanwhile in 2014 correlation was not significant. At both consecutive years, certified rheumatologist had significantly higher OSCE scores than TRs. There were more TRs with a MCQ pass score among TRs with an OSCE pass score than among TRs with an OSCE failing score: 86% vs. 67%, p=0.02. TRs with an OSCE failing score were more frequently distributed in the bottom 2 quartiles on the MCQ (p=0.07).
Nine stations were applied at 2013 and 2014 OSCE circuits, and their (mean±SD) scores showed good correlation, r from 0.81 to 0.95, p≤0.01.
Conclusion
The OSCE was a valid and reliable tool to assess clinical skill competency in TRs.
Disclosure:
V. Pascual Ramos,
None;
G. Medrano-Ramírez,
None;
E. Solis-Vallejo,
None;
A. Bernard-Medina,
None;
D. Flores,
None;
M. Portela Hernandez,
None;
L. Andrade-Ortega,
None;
O. L. Vera-Lastra,
None;
R. Espinosa-Morales,
None;
J. Miranda-Limón,
None;
M. Maldonado-Velázquez,
None;
L. J. Jara,
None;
L. M. Amezcua-Guerra,
None;
J. Lopez-Zepeda,
None;
M. A. Saavedra,
None;
C. A. Arce,
None.
« Back to 2014 ACR/ARHP Annual Meeting
ACR Meeting Abstracts - https://acrabstracts.org/abstract/implementation-and-performance-of-an-objective-structured-clinical-examination-osce-in-a-national-certification-process-of-trainees-in-rheumatology-two-years-of-experience/