Background/Purpose: The American College of Rheumatology (ACR) practice guidelines establish U.S. and international treatment recommendations and hence performance measures and research priorities, so we sought to characterize the quality of evidence, the strength of recommendations and their relationship in the ACR guidelines.

Methods: For consistency (because ACR guidelines as of March 2016 used 3 different evidence grading systems), two independent reviewers applied the American College of Cardiology (ACC)/ American Heart Association (AHA) grading system to evaluate the quality (level) of evidence and class (benefit to harm or strength) of ACR treatment recommendations. To give each guideline equal weight, we reported results as medians and interquartile ranges (IQR) of the percentages of recommendations by level and class within each guideline.

Results: In the 8 guidelines (Glucocorticoid Induced Osteoporosis (GIOP); Juvenile Idiopathic Arthritis (JIA); Gout;  Lupus Nephritis (LN); Osteoarthritis (OA); Spondyloarthritis (SpA); Polymyalgia Rheumatica (PMR); Rheumatoid Arthritis (RA)) involving 403 (range 10-102) recommendations, 23% (IQR 12-30%) were supported by level A (high) quality evidence; 18% (IQR 15-20%) level B (moderate); and 50% level C (IQR 46-70%) (low) (Table 1).  In the 4 guidelines that reported the strength of 143 recommendations, 26% (IQR 21-31%) were class I (high benefit to harm ratio); 62% (IQR 58-64%) class II (moderate to low benefit to harm); and 16% (IQR 13-19%) class III (harm exceeds benefit) (Table 2). Overall, only 10% (IQR 7-12%) of recommendations (mean 9% of 143 recommendations) had both level A evidence and class I strength of recommendation. Of the level A and class I recommendations, the majority involved OA. Similarly, only 6% (IQR 2-12%) of recommendations (mean 9% of 143 recommendations) had both level B evidence and class I strength of recommendation. Most recommendations were level C and class II (median 30%, IQR 21-41%%, mean 32%). At the extremes, OA (58%) and GIOP (35%) had the most level A recommendations, and JIA (2%) and PMR (10%) had the least. Conversely, level C evidence comprised 86% of JIA recommendations, 70% of RA and 70% of LN.

Conclusion: The ACR clinical practice guidelines consist primarily of low-level evidence with moderate to low strength of recommendation. Few recommendations are both strong and supported by high-level evidence, emphasizing the need to expand the evidence base.