Making sense of patient-reported outcomes when making recommendations

Applying results of clinical trials in the context of making treatment recommendations presents many challenges.  In this article, we describe how we addressed the challenge of interpreting results in a guideline addressing arthroscopy in patients with degenerative knee disease.
Investigators increasingly rely on patient-reported outcomes (PROs) as key endpoints in clinical trials. Although PROs provide patients’ experience of the impact of disease and treatment on their health status, challenges in interpreting changes in PRO scores can limit their usefulness in informing patient-centered care.
A key issue for those making recommendations on the basis of clinical trials using PROs is how patients value the outcomes: where in the continuum between trivial and very important will patients place observed improvements in PROs such as pain or physical function? Knowledge of the minimal important difference (MID), the smallest change that patients perceive as important, either beneficial or harmful, facilitates an understanding of the magnitude of intervention effects in randomized trials.
The MAGIC, non-profit research and innovation programme – representing patients, front-line clinicians, researchers, and guideline experts (www.magicproject.org) has recently partnered with the BMJ to publish trustworthy recommendations in response to potentially practice changing evidence: BMJ Rapid Recommendations1. BMJ Rapid Recommendations panels, as in any guideline, require appropriate interpretation of the importance of effects when moving from evidence to recommendations – judgments that should reflect patients’ values and preferences. The panel responsible for creating the second BMJ Rapid Recommendation, addressing the impact of arthroscopic surgery versus non-operative management in patients with degenerative knee disease, faced challenges in interpreting the significance of apparent treatment effects on critical outcomes of interest: pain, function, and quality of life (QoL).

To address this challenge, we conducted a systematic review to identify the most trustworthy MID estimates for the PROs used in trials comparing arthroscopic surgery to conservative management.
We identified 13 studies, many of which suffered from serious methodologic limitations, that reported on 95 empirically estimated anchor-based MIDs for 8 PRO instruments and/or their sub-domains that measure knee pain, function or QoL. We identified credible MIDs for the Western Ontario McMaster Arthritis Index (WOMAC), Knee injury Osteoarthritis Outcome Score (KOOS) and the EuroQol five dimensions questionnaire (EQ-5D).
Our systematic review showed that MIDs may vary substantially by estimation method, population and context. We were able to distinguish between more and less trustworthy MIDs and provide best estimates for key instruments that informed evidence presentation in the associated systematic review of treatment effects, and judgments in the Rapid Recommendation. The panel, aware through use of the MID that benefits associated with arthroscopy were very small, made a strong recommendation against knee arthroscopy.
Though we were able to distinguish the more or less trustworthy MIDs, the range of estimates among those deemed credible was still very wide.  At the time of writing, we are in negotiation with the BMJ regarding the fate of our review of MIDs, the associated systematic review, and the recommendation itself.
Our study provides a model for applying the MID concept to aid in the interpretation of evidence, and the formulation of recommendations for clinical practice guidelines, and highlights the challenges when trustworthy MIDs are not available.
Our group is currently conducting several projects to advance MID methods including the development of a definitive credibility instrument and testing its reliability, a comprehensive systematic survey of the MID methods literature, a systematic review to identify anchor-based MIDs for all known PRO instruments and, in collaboration with the Cochrane PRO methods group, a systematic survey of PRO aggregation methods employed in Cochrane reviews.  We welcome collaboration by anyone interested in this work.
References

Siemieniuk RA, Agoritsas T, Macdonald H, et al. Introduction to BMJ Rapid Recommendations. BMJ (Clinical research ed) 2016;354:i5191. doi: 10.1136/bmj.i5191 [published Online First: 2016/09/30]