Skip to main content
SpringerLink
Log in

Methods for interpreting change over time in patient-reported outcome measures

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Interpretation guidelines are needed for patient-reported outcome (PRO) measures’ change scores to evaluate efficacy of an intervention and to communicate PRO results to regulators, patients, physicians, and providers. The 2009 Food and Drug Administration (FDA) Guidance for Industry Patient-Reported Outcomes (PRO) Measures: Use in Medical Product Development to Support Labeling Claims (hereafter referred to as the final FDA PRO Guidance) provides some recommendations for the interpretation of change in PRO scores as evidence of treatment efficacy.

Methods

This article reviews the evolution of the methods and the terminology used to describe and aid in the communication of meaningful PRO change score thresholds.

Results

Anchor- and distribution-based methods have played important roles, and the FDA has recently stressed the importance of cross-sectional patient global assessments of concept as anchor-based methods for estimation of the responder definition, which describes an individual-level treatment benefit. The final FDA PRO Guidance proposes the cumulative distribution function (CDF) of responses as a useful method to depict the effect of treatments across the study population.

Conclusions

While CDFs serve an important role, they should not be a replacement for the careful investigation of a PRO’s relevant responder definition using anchor-based methods and providing stakeholders with a relevant threshold for the interpretation of change over time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Abbreviations

AQLQ:

Asthma Quality of Life Questionnaire

CDF:

Cumulative distribution function

CHQ:

Chronic Heart Failure Questionnaire

CRQ:

Chronic Respiratory Questionnaire

ECOG:

Eastern Cooperative Oncology Group

ES:

Effect size

FDA:

Food and drug administration

IAC:

Industry Advisory Committee

ISOQOL:

International Society for Quality of Life Research

MCID:

Minimal clinically important difference

MID:

Minimal important difference

PRO:

Patient-reported outcome

QOL:

Quality of life

References

  1. Patient-Centered Outcomes Research Institute (PCORI). Available at: http://www.pcori.org/home.html.

  2. King, M. T. (2011). A point of minimal important difference (MID): A critique of terminology and methods. Expert review of pharmacoeconomics & outcomes research, 11(2), 171–184.

    Article  Google Scholar 

  3. Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 74(235), 65132–65133.

    Google Scholar 

  4. Burke, L. B., & Trenacosti, A. M. (2010). Interpretation of PRO trial results to support FDA labelling claims: the regulator perspective. International Society for Pharmacoecomomics and Outcomes Research 15th Annual International Meeting. Atlanta: GA.

  5. Jaeschke, R., Singer, J., & Guyatt, G. H. (1989). Measurement of health status. Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10(4), 407–415.

    Article  PubMed  CAS  Google Scholar 

  6. Guyatt, G. H., Berman, L. B., & Townsend, M. (1987). Long-term outcome after respiratory rehabilitation. Canadian Medical Association Journal, 137(12), 1089–1095.

    PubMed  CAS  Google Scholar 

  7. Guyatt, G. H., Townsend, M., Nogradi, S., Pugsley, S. O., Keller, J. L., & Newhouse, M. T. (1988). Acute response to bronchodilator. An imperfect guide for bronchodilator therapy in chronic airflow limitation. Archives of Internal Medicine, 148(9), 1949–1952.

    Article  PubMed  CAS  Google Scholar 

  8. Guyatt, G. H., Sullivan, M. J., Fallen, E. L., Tihal, H., Rideout, E., Halcrow, S., et al. (1988). A controlled trial of digoxin in congestive heart failure. American Journal of Cardiology, 61(4), 371–375.

    Article  PubMed  CAS  Google Scholar 

  9. Guyatt, G. H., Berman, L. B., Townsend, M., Pugsley, S. O., & Chambers, L. W. (1987). A measure of quality of life for clinical trials in chronic lung disease. Thorax, 42(10), 773–778.

    Article  PubMed  CAS  Google Scholar 

  10. Guyatt, G. H., Nogradi, S., Halcrow, S., Singer, J., Sullivan, M. J., & Fallen, E. L. (1989). Development and testing of a new measure of health status for clinical trials in heart failure. Journal of General Internal Medicine, 4(2), 101–107.

    Article  PubMed  CAS  Google Scholar 

  11. Juniper, E. F., Guyatt, G. H., Willan, A., & Griffith, L. E. (1994). Determining a minimal important change in a disease-specific Quality of Life Questionnaire. Journal of Clinical Epidemiology, 47(1), 81–87.

    Article  PubMed  CAS  Google Scholar 

  12. Sloan, J. A., Cella, D., Frost, M., Guyatt, G. H., Sprangers, M., & Symonds, T. (2002). Assessing clinical significance in measuring oncology patient quality of life: Introduction to the symposium, content overview, and definition of terms. Mayo Clinic Proceedings, 77(4), 367–370.

    Article  PubMed  Google Scholar 

  13. Guyatt, G. H., Osoba, D., Wu, A. W., Wyrwich, K. W., & Norman, G. R. (2002). Methods to explain the clinical significance of health status measures. Mayo Clinic Proceedings, 77(4), 371–383.

    Article  PubMed  Google Scholar 

  14. Cella, D., Bullinger, M., Scott, C., & Barofsky, I. (2002). Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clinic Proceedings, 77(4), 384–392.

    Article  PubMed  Google Scholar 

  15. Sloan, J. A., Aaronson, N., Cappelleri, J. C., Fairclough, D. L., & Varricchio, C. (2002). Assessing the clinical significance of single items relative to summated scores. Mayo Clinic Proceedings, 77(5), 479–487.

    PubMed  Google Scholar 

  16. Frost, M. H., Bonomi, A. E., Ferrans, C. E., Wong, G. Y., & Hays, R. D. (2002). Patient, clinician, and population perspectives on determining the clinical significance of quality-of-life scores. Mayo Clinic Proceedings, 77(5), 488–494.

    PubMed  Google Scholar 

  17. Sprangers, M. A., Moinpour, C. M., Moynihan, T. J., Patrick, D. L., & Revicki, D. A. (2002). Assessing meaningful change in quality of life over time: A users’ guide for clinicians. Mayo Clinic Proceedings, 77(6), 561–571.

    Article  PubMed  Google Scholar 

  18. Symonds, T., Berzon, R., Marquis, P., & Rummans, T. A. (2002). The clinical significance of quality-of-life results: Practical considerations for specific audiences. Mayo Clinic Proceedings, 77(6), 572–583.

    Article  PubMed  Google Scholar 

  19. Food and Drug Administration. (2006). Draft guidance for industry on patient-reported outcome measures: Use in medical product development to support labeling claims. Federal Register, 71(23), 5862–5863.

    Google Scholar 

  20. Hamilton, M. (1967). Development of a rating scale for primary depressive illness. The British Journal of Social and Clinical Psychology, 6(4), 278–296.

    Article  PubMed  CAS  Google Scholar 

  21. Revicki, D. A., Erickson, P. A., Sloan, J. A., Dueck, A., Guess, H., & Santanello, N. C. (2007). Interpreting and reporting results based on patient-reported outcomes. Value Health, 10(Suppl 2), S116–S124.

    Article  PubMed  Google Scholar 

  22. Patrick, D. L., Burke, L. B., Powers, J. H., Scott, J. A., Rock, E. P., Dawisha, S., et al. (2007). Patient-reported outcomes to support medical product labeling claims: FDA perspective. Value Health, 10(Suppl 2), S125–S137.

    Article  PubMed  Google Scholar 

  23. Lydick, E., & Epstein, R. S. (1993). Interpretation of quality of life changes. Quality of Life Research, 2(3), 221–226.

    Article  PubMed  CAS  Google Scholar 

  24. Revicki, D., Hays, R. D., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109.

    Article  PubMed  Google Scholar 

  25. Sloan, J. A., Frost, M. H., Berzon, R., Dueck, A., Guyatt, G., Moinpour, C., et al. (2006). The clinical significance of quality of life assessments in oncology: A summary for clinicians. Supportive Care in Cancer, 14(10), 988–998.

    Article  PubMed  Google Scholar 

  26. Farrar, J. T., Young, J. P., Jr, LaMoreaux, L., Werth, J. L., & Poole, R. M. (2001). Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain, 94(2), 149–158.

    Article  PubMed  CAS  Google Scholar 

  27. Norman, G. R., Stratford, P., & Regehr, G. (1997). Methodological problems in the retrospective computation of responsiveness to change: The lesson of Cronbach. Journal of Clinical Epidemiology, 50(8), 869–879.

    Article  PubMed  CAS  Google Scholar 

  28. Walters, S. J., & Brazier, J. E. (2005). Comparison of the minimally important difference for two health state utility measures: EQ-5D and SF-6D. Quality of Life Research, 14(6), 1523–1532.

    Article  PubMed  Google Scholar 

  29. Metz, S. M., Wyrwich, K. W., Babu, A. N., Kroenke, K., Tierney, W. M., & Wolinsky, F. D. (2007). Validity of patient-reported health-related quality of life global ratings of change using structural equation modeling. Quality of Life Research, 16(7), 1193–1202.

    Article  PubMed  Google Scholar 

  30. Wyrwich, K., Harnam, N., Revicki, D. A., Locklear, J. C., Svedsater, H., & Endicott, J. (2009). Assessing health-related quality of life in generalized anxiety disorder using the Quality Of Life Enjoyment and Satisfaction Questionnaire. International Clinical Psychopharmacology, 24(6), 289–295.

    Article  PubMed  Google Scholar 

  31. Brozek, J. L., Guyatt, G. H., & Schunemann, H. J. (2006). How a well-grounded minimal important difference can enhance transparency of labelling claims and improve interpretation of a patient reported outcome measure. Health and Quality of Life Outcomes, 4, 69.

    Article  PubMed  Google Scholar 

  32. Kosinski, M., Zhao, S. Z., Dedhiya, S., Osterhaus, J. T., & Ware, J. E., Jr. (2000). Determining minimally important changes in generic and disease-specific health-related quality of life questionnaires in clinical trials of rheumatoid arthritis. Arthritis and Rheumatism, 43(7), 1478–1487.

    Article  PubMed  CAS  Google Scholar 

  33. Eton, D. T., Cella, D., Yost, K. J., Yount, S. E., Peterman, A. H., Neuberg, D. S., et al. (2004). A combination of distribution- and anchor-based approaches determined minimally important differences (MIDs) for four endpoints in a breast cancer scale. Journal of Clinical Epidemiology, 57(9), 898–910.

    Article  PubMed  Google Scholar 

  34. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  35. Kazis, L. E., Anderson, J. J., & Meenan, R. F. (1989). Effect sizes for interpreting changes in health status. Medical Care, 27(3 Suppl), S178–S189.

    Article  PubMed  CAS  Google Scholar 

  36. Norman, G. R., Wyrwich, K. W., & Patrick, D. L. (2007). The mathematical relationship among different forms of responsiveness coefficients. Quality of Life Research, 16(5), 815–822.

    Article  PubMed  CAS  Google Scholar 

  37. Liang, M. H. (1995). Evaluating measurement responsiveness. Journal of Rheumatology, 22(6), 1191–1192.

    PubMed  CAS  Google Scholar 

  38. Norman, G. R., Sloan, J. A., & Wyrwich, K. W. (2003). Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Medical Care, 41(5), 582–592.

    PubMed  Google Scholar 

  39. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory. New York: McGraw-Hill.

    Google Scholar 

  40. Wyrwich, K. W., Tierney, W. M., & Wolinsky, F. D. (1999). Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. Journal of Clinical Epidemiology, 52(9), 861–873.

    Article  PubMed  CAS  Google Scholar 

  41. Wyrwich, K. W. (2004). Minimal important difference thresholds and the standard error of measurement: Is there a connection? Journal of Biopharmaceutical Statistics, 14(1), 97–110.

    Article  PubMed  Google Scholar 

  42. Wyrwich, K. W., Tierney, W. M., & Wolinsky, F. D. (2002). Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire. Quality of Life Research, 11(1), 1–7.

    Article  PubMed  Google Scholar 

  43. Cella, D., Eton, D. T., Fairclough, D. L., Bonomi, P., Heyes, A. E., Silberman, C., et al. (2002). What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire? Results from Eastern Cooperative Oncology Group (ECOG) Study 5592. Journal of Clinical Epidemiology, 55(3), 285–295.

    Article  PubMed  Google Scholar 

  44. Crosby, R. D., Kolotkin, R. L., & Williams, G. R. (2004). An integrated method to determine meaningful changes in health-related quality of life. Journal of Clinical Epidemiology, 57(11), 1153–1160.

    Article  PubMed  Google Scholar 

  45. Yost, K. J., Cella, D., Chawla, A., Holmgren, E., Eton, D. T., Ayanian, J. Z., et al. (2005). Minimally important differences were estimated for the Functional Assessment of Cancer Therapy-Colorectal (FACT-C) instrument using a combination of distribution- and anchor-based approaches. Journal of Clinical Epidemiology, 58(12), 1241–1251.

    Article  PubMed  CAS  Google Scholar 

  46. ARICEPT Oral Solution (Donepezil Hydrochloride) [approval label]. Available at: http://www.accessdata.fda.gov/drugsatfda_docs/label/2004/21719lbl.pdf.

  47. Copay, A. G., Subach, B. R., Glassman, S. D., Polly, D. W., Jr, & Schuler, T. C. (2007). Understanding the minimum clinically important difference: A review of concepts and methods. Spine Journal, 7(5), 541–546.

    Article  PubMed  Google Scholar 

  48. Sprangers, M. A., & Schwartz, C. E. (1999). Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine, 48(11), 1507–1515.

    Article  PubMed  CAS  Google Scholar 

  49. Rapkin, B. D., & Schwartz, C. E. (2004). Toward a theoretical model of quality-of-life appraisal: Implications of findings from studies of response shift. Health and Quality of Life Outcomes, 2, 14.

    Article  PubMed  Google Scholar 

  50. Barclay-Goddard, R., Epstein, J. D., & Mayo, N. E. (2009). Response shift: A brief overview and proposed research priorities. Quality of Life Research, 18(3), 335–346.

    Article  PubMed  Google Scholar 

  51. Sprangers, M. A., & Aaronson, N. K. (1992). The role of health care providers and significant others in evaluating the quality of life of patients with chronic disease: A review. Journal of Clinical Epidemiology, 45(7), 743–760.

    Article  PubMed  CAS  Google Scholar 

  52. von Essen, L. (2004). Proxy ratings of patient quality of life–factors related to patient-proxy agreement. Acta Oncologica, 43(3), 229–234.

    Article  Google Scholar 

  53. van der Linden, F. A., Kragt, J. J., van Bon, M., Klein, M., Thompson, A. J., van der Ploeg, H. M., et al. (2008). Longitudinal proxy measurements in multiple sclerosis: Patient-proxy agreement on the impact of MS on daily life over a period of two years. BMC Neurol, 8, 2.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Members of the Industry Advisory Committee (IAC), the Board of Directors of the International Society for Quality of Life Research (ISOQOL), and two anonymous reviewers offered valuable suggestions that were incorporated into this paper.

Author information

Authors and Affiliations

  1. United BioSource Corporation, 7101 Wisconsin Avenue, Suite 600, Bethesda, MD, 20814, USA

    K. W. Wyrwich & W. R. Lenderking

  2. Merck Sharp & Dohme, Inc., North Wales, PA, USA

    J. M. Norquist

  3. Oxford Outcomes Ltd, Oxford, UK

    S. Acaster

Authors
  1. K. W. Wyrwich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. M. Norquist
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. W. R. Lenderking
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. S. Acaster
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

the Industry Advisory Committee of International Society for Quality of Life Research (ISOQOL)

Corresponding author

Correspondence to K. W. Wyrwich.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wyrwich, K.W., Norquist, J.M., Lenderking, W.R. et al. Methods for interpreting change over time in patient-reported outcome measures. Qual Life Res 22, 475–483 (2013). https://doi.org/10.1007/s11136-012-0175-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-012-0175-x

Keywords

Search

Navigation