Special communication
Applications of Response Shift Theory and Methods to Participation Measurement: A Brief History of a Young Field

https://doi.org/10.1016/j.apmr.2009.11.029Get rights and content

Abstract

Schwartz CE. Applications of response shift theory and methods to participation measurement: a brief history of a young field.

Measurement of participation in people with disability can pose psychometric and conceptual challenges. Ambiguous or paradoxical findings can occur because of differences among people or changes within people regarding internal standards, values, or conceptualization of participation. These response shifts can affect standard psychometric indices, such as reliability and validity. We focus herein on the interpretation of patient-reported outcomes and, in particular, on the cognitive appraisal processes known as response shift. We present theoretical and conceptual distinctions building on response shift theory and other current developments in health-related quality of life research to inform participation measurement research. We discuss how response shifts can influence the interpretation of reliability, validity, and responsiveness of participation measures. We then discuss the evidence for the clinical significance of response shift phenomena and describe current design, statistical, and individualized approaches for detecting response shift phenomena.

Section snippets

Response Shift Phenomena and Participation

When individuals experience a health state change, they may change their internal standards (ie, recalibration), values (ie, reprioritization), or meaning (ie, reconceptualization) of the target construct one is asking them to self-report, such as participation. For example, people with a substantial physical disability may experience a severity of fatigue that they did not know prior to the development of the disability. Consequently, they would recalibrate what severe fatigue means to them,

What Do Psychometric Characteristics Mean in the Context of Response Shift?

Indeed, one of the most challenging aspects of response shift research is that it calls into question fundamental assumptions of questionnaires (eg, measurement invariance) and psychometric criteria such as reliability, validity, and responsiveness. Schwartz and Rapkin4 noted that every quantitative index of reliability, validity, and responsiveness may be distorted by reasonable and expected adaptation-related changes over time. For example, high internal consistency (reliability) and

A Brief History of Response Shift Theory

The study of response shift began formally in the fields of management science and educational research and was exported to the field of health-related QOL research in the mid-1990s. In 1999, a theoretical model was described by Sprangers and Schwartz2 that provided a framework for thinking about the meaning of response shift in clinical research. It was proposed that after a change in health state (the catalyst), stable personality characteristics (ie, antecedents) interact with cognitive,

Does It Matter? The Clinical Significance of Response Shift Phenomena

Recent efforts in the field of QOL research have focused on the interpretation of changes in patient-reported outcome scores. The recognition that statistical significance does not necessarily mean clinical importance has led to the development of a host of indices of clinical importance (eg, the minimal clinically important difference; see reference21 for overview of this subfield). Investigators are also increasingly required by professional journals to express their research findings in

Methodologic Advances in Response Shift Detection

Current methods for detecting response shifts are evolving from a predominant focus on the then-test approach to an emphasis on statistical and individualized methods. As noted, the then-test approach has the advantage of being easy to administer and analyze but the disadvantages of random error and/or confounding with recall bias as well as difficulty of interpretation. For these reasons, we briefly describe promising statistical and individualized methods that have evolved in the past few

Conclusions

Response shift phenomena are cognitive appraisal processes that focus on the interpretation of patient-reported outcomes and, as such, are salient in participation measurement. QOL response shift has been evaluated in a number of populations characterized by disability (eg, multiple sclerosis, stroke, cancer, musculoskeletal diseases), but the focus on participation response shift has yet to occur. It is our hope that this overview will facilitate research into participation response shift.

References (48)

  • M.R. Visser et al.

    How response shift may affect the measurement of change in fatigue

    J Pain Symptom Manage

    (2000)
  • S. Nolte et al.

    A test of measurement invariance fails to support the application of then-test questions as a remedy to response shift bias

    J Clin Epidemiol

    (2009)
  • S. Ahmed et al.

    The Oort structural equation modeling approach detected a response shift after a COPD self-management program not detected by the Schmitt technique

    J Clin Epidemiol

    (2009)
  • R. Barclay-Goddard et al.

    Response shift was identified over multiple occasions with a structural equation modeling framework

    J Clin Epidemiol

    (2009)
  • N. Mayo et al.

    Case management post-stroke did not induce response shift: the value of residuals

    J Clin Epidemiol

    (2009)
  • C.A. O'Boyle et al.

    Individual quality of life in patients undergoing hip replacement

    Lancet

    (1992)
  • M. Brown et al.

    Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives

    J Head Trauma Rehabil

    (2004)
  • C.E. Schwartz et al.

    Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal

    Health Qual Life Outcomes

    (2004)
  • B.D. Rapkin et al.

    Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift

    Health Qual Life Outcomes

    (2004)
  • Aristotle. Nicomachean ethics 384-322...
  • Plato. The Republic, Book IX. c.380...
  • R.D. Hays et al.

    Responsiveness to change: an aspect of validity, not a separate dimension

    Qual Life Res

    (1992)
  • H. Helson

    Adaptation level theory

    (1964)
  • S. Folkman et al.

    Positive meaningful events and coping in the context of HIV/AIDS

  • Cited by (58)

    • Promoting a relational approach to residential child care through an organizational program model: Impacts of CARE implementation on staff outcomes

      2022, Children and Youth Services Review
      Citation Excerpt :

      Some factors are beyond the control of practitioners and agency administrators, such as economic conditions that can reduce the available workforce and reduce staffing ratios (e.g., Baker, Dixon, & Passmore, 2010), or regulatory changes that influence who becomes placed, and for how long (Bellonci, Holmes, & Whittaker, 2019). In addition, the change observed in this variable may have been limited by response-shift (e.g. Schwartz, 2010), a phenomenon in which an intervention can lead to lower post-test ratings by changing how participants evaluate survey questions (i.e., perhaps leading them to set a higher standard for themselves after learning more about the practices included on the survey). In addition, the sensitivity of the Practice measure may have been reduced because of its focus on relational practices that involve some form of positive engagement and the absence of questions about practices that are punitive, controlling, or lacking sensitivity to children’s needs.

    View all citing articles on Scopus

    Presented to the American Congress of Rehabilitation Medicine, Symposium on Measurement in Participation, October 14–15, 2008, Toronto, ON, Canada.

    Supported by an honorarium by the Rehabilitation Institute of Chicago using funds received from the National Institute on Disability and Rehabilitation Research (grant no. H133B040032).

    No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.

    View full text