Background/Purpose:

Understanding epidemiology and biostatistics (epi/biostats) is crucial for rheumatologists to interpret literature and make appropriate data-driven clinical decisions. Based on retrieval-enhanced learning theory, we developed CLASS-Rheum, a cloud-based modular assessment tool to support learning of epi/biostats relevant to rheumatology fellows. In this study we test whether sequential administration of isomorphic Question Sets is associated with sustained learning.

Methods:

CLASS-Rheum Question Sets are organized into 10 modules, each focused on a concept in epi/biostats, framed in rheumatologic context. Administration via Learning Catalytics® allows access on web-enabled devices. With IRB-exemption, 6 rheumatology programs of varying size across the US participated in sequential administration of 2 Question Sets. Question Set 2 (QS2) was comprised of 56 unique questions assessing the same content and learning objectives as Question Set 1 (QS1). We examined face and construct validity, item difficulty, psychometrics, and change in knowledge with sequential administration of Question Sets. Fellows providing individual responses to > 50% of questions were included in the quantitative analysis. Mean change in percent correct and change in Likert-scale ratings were compared with paired-t tests and Wilcoxon Signed-Rank tests, respectively. Mixed effects modeling was used to examine whether year of fellowship impacted percent correct (SAS v9.4).

Results:

From 2/2016-5/2017, 51 rheumatology fellows enrolled in CLASS-Rheum. 39 provided individual responses to >50% of questions and were included in the quantitative analysis. 67% of participating fellows were women. At enrollment, 10% had degrees in epi/biostats, 61% reported being very interested in epi/biostats, and 65% considered their understanding of epi/biostats average compared to other rheumatology fellows. 28 fellows’ data were analyzed in QS1 (all first time takers), and 25 fellows’ in QS2 (11 first time takers). 14 fellows completed both Question Sets, with mean interval between pairs of isomorphic modules of 260 days (SD 70.7).

Difficulty was similar between modules. Mean percent correct for first time takers was 64.4% (SD 15.1%) for QS1 vs. 64.8 (SD 11.9%) for QS2 (p = 0.93). Point biserial correlations were positive for 93% of questions from QS1 and QS2, suggesting strong psychometrics. Based on data from QS1 and QS2, 3^rd year fellows had statistically higher average scores than 1^st year fellows (p=0.032).

Among the 14 fellows completing QS1 and QS2, mean percent correct increased from 61.8% to 67.9% (p = 0.036). 14% of fellows’ scores increased >20%. Upon completing QS1, 14/14 fellows described CLASS-Rheum as “very useful/useful” in learning epi/biostats, as did 13/14 after completing QS2 (p=0.75).

Conclusion:

Two psychometrically sound modular Question Sets (“CLASS-Rheum”) covering important concepts in epi/biostats, framed in a rheumatologic context, were successfully administered to fellows in diverse rheumatology training programs. Sustained increases in knowledge were demonstrated with sequential administration of isomorphic Question Sets over 15 months. CLASS-Rheum will be evaluated in additional programs.

« Back to 2017 ACR/ARHP Annual Meeting

ACR Meeting Abstracts - https://acrabstracts.org/abstract/the-class-rheum-critical-literature-assessment-skills-support-rheumatology-question-based-tool-is-associated-with-sustained-improvement-in-knowledge-of-relevant-epidemiology-and-biostatist/