Background/Purpose:
Value of traditional pre- and post-course assessments is uncertain. Having learners re-evaluate pre-course competency after a course may be a more useful tool for curriculum evaluation. This project explores whether “retrospective” self-assessments provide a more useful measure of program effectiveness than traditional pre- and post-course assessments.
Methods:
The Department of Veterans Affairs created a program in musculoskeletal care for primary providers. To date, this has involved 130 participants across six national sites. Prior to the course, providers used a 5-point Likert scale to rate proficiency in approaching shoulder, knee, and back pain. Curriculum included focused didactics and small group hands-on practice sessions with simulated patients for shoulder and knee exams. In contrast, back pain was addressed through a single didactic session. After the program, providers retrospectively rated pre-course proficiency on the same dimensions, as well as post-course proficiency. Paired t-tests compared mean prospective and retrospective assessments.
Results:
Post-course ratings were higher than pre-course ratings across all items. Retrospective ratings of pre-course competency in assessing shoulder and knee pain were significantly lower than initial, prospective pre-course ratings. In contrast, retrospective ratings for competency with back pain were not significantly lowered.
|
|
Mean Pre-course Ratings |
Mean Paired Difference (SEM; p) |
Mean Post-course Ratings |
Pre-Post Change |
||
|
Prospective |
Retrospective |
Post-Pro |
Post-Retro |
|||
|
Shoulder Pain |
|
|
|
|
|
|
|
I can examine and diagnose shoulder pain without MRI |
3.1 |
2.7 |
0.3 (0.09; 0.001) |
4.7 |
1.6 |
2.0 |
|
I can evaluate patients effectively |
3.2 |
2.8 |
0.3 (0.08; <0.001) |
4.8 |
1.6 |
2.0 |
|
I can develop a appropriate plan |
3.3 |
3.0 |
0.2 (0.09; 0.011) |
4.7 |
1.4 |
1.7 |
|
I understand when to order imaging |
3.4 |
3.1 |
0.3 (0.09; 0.003) |
4.8 |
1.4 |
1.7 |
|
I understand when to refer |
3.6 |
3.3 |
0.2 (0.08; 0.007) |
4.7 |
1.1 |
1.4 |
|
Knee Pain |
|
|
|
|
|
|
|
I can examine and diagnose knee pain without MRI |
3.2 |
2.9 |
0.3 (0.08; 0.01) |
4.7 |
1.5 |
1.8 |
|
I can evaluate patients effectively |
3.3 |
3.0 |
0.2 (0.07; 0.01) |
4.7 |
1.4 |
1.7 |
|
I can develop an appropriate plan |
3.5 |
3.1 |
0.3 (0.08; 0.01) |
4.6 |
1.1 |
1.5 |
|
I understand when to order imaging |
3.6 |
3.2 |
0.2 (0.08; 0.03) |
4.7 |
1.1 |
1.5 |
|
I understand when to refer |
3.7 |
3.3 |
0.3 (0.09; 0.01) |
4.7 |
1.0 |
1.4 |
|
Back Pain |
|
|
|
|
|
|
|
I can identify patients with low back for whom MRI is appropriate |
3.8 |
3.6 |
0.04 (0.09; 0.65) |
4.5 |
0.7 |
0.9 |
|
I can develop a reasonable management plan |
3.8 |
3.7 |
0.01 (0.09; 0.91) |
4.5 |
0.7 |
0.8 |
|
I understand when to refer |
3.8 |
3.7 |
0.07 (0.09; 0.42) |
4.6 |
0.8 |
0.9 |
Conclusion:
Lower retrospective pre-course ratings were seen only in course elements that involved multiple methods of instruction. Multi-modal skill acquisition may lead to a more critical assessment pre-course proficiency. Incorporating retrospective pre-course self-assessment may distinguish educational programs that most effectively teach new skills.
Disclosure:
M. J. Battistone,
None;
A. M. Barker,
None;
J. P. Beck,
None;
M. Grotzke,
None;
T. A. Huhtala,
None;
J. Butler,
None;
A. C. Cannella,
None;
D. I. Daikh,
None;
M. A. Fang,
None;
A. A. Lazzari,
None;
P. Roldan,
None;
J. M. Von Feldt,
None;
G. W. Cannon,
None.
« Back to 2013 ACR/ARHP Annual Meeting
ACR Meeting Abstracts - https://acrabstracts.org/abstract/retrospective-self-assessment-of-pre-course-competency-a-useful-tool-for-musculoskeletal-curriculum-assessment-in-a-multi-center-interprofessional-cohort/