Methods in Cardiovascular Research
A Review of Propensity-Score Methods and Their Use in Cardiovascular Research

https://doi.org/10.1016/j.cjca.2015.05.015Get rights and content

Abstract

Observational studies using propensity-score methods have been increasing in the cardiovascular literature because randomized controlled trials are not always feasible or ethical. However, propensity-score methods can be confusing, and the general audience may not fully understand the importance of this technique. The objectives of this review are to describe (1) the fundamentals of propensity score methods, (2) the techniques to assess for propensity-score model adequacy, (3) the 4 major methods for using the propensity score (matching, stratification, covariate adjustment, and inverse probability of treatment weighting [IPTW]) using examples from previously published cardiovascular studies, and (4) the strengths and weaknesses of these 4 techniques. Our review suggests that matching or IPTW using the propensity score have shown to be most effective in reducing bias of the treatment effect.

Résumé

Les études observationnelles utilisant des méthodes de score de propension sont en augmentation dans les publications à thématique cardiovasculaire car les essais randomisés contrôlés ne sont pas toujours réalisables ou éthiques. Cependant, les méthodes de score de propension peuvent être source de confusion, et le public peut ne pas pleinement comprendre l'importance de cette méthode. Les objectifs de cette revue sont de décrire (1) les principes fondamentaux des méthodes de score de propension, (2) les méthodes pour évaluer l’adéquation du modèle de score de propension, (3) les 4 principales méthodes d’utilisation du score de propension (appariement, stratification, ajustement de covariable, et probabilité inverse du traitement pondéré [PITP]) en utilisant des exemples provenant d'études cardiovasculaires publiées précédemment, et (4) les forces et les faiblesses de ces 4 méthodes. Notre revue suggère que l'appariement ou la PITP utilisant le score de propension se sont montrés plus efficients dans la réduction du biais de l'effet du traitement.

Section snippets

Definition and Fundamentals of the Propensity Score

Rosenbaum and Rubin defined the propensity score as the probability (from 0-1) of treatment assignment based on observed baseline covariates.5 Thus, it is the conditional probability that a participant will be assigned to the treatment group based on patient demographics and comorbidities that are measured at the time of group assignment.2, 4 The propensity score can be thought of as a balancing score that, like random assignment, attempts to balance the distribution of these measured

Assessing for Model Adequacy

The success of propensity-score modelling can be evaluated by comparing the balance between treated and control participants after accounting for the propensity score; if there are imbalances, especially for prognostically important covariates, the model has been inadequately specified.13 Once a propensity-score model has been created, appropriate methods should be used to examine whether the model does indeed lead to similar distribution of covariates (ie, correctly specified) between treated

Propensity-Score Matching

Propensity-score matching is 1 of the most common ways to use the propensity score. It is a widely used method in the cardiovascular literature, although its implementation and reporting have been found to be poor.11 In propensity-score matching, participants with similar propensity scores in the treatment and control groups are matched.

An example of this technique was demonstrated in a study by Tu et al.8 comparing the effect of drug-eluting stents vs bare-metal stents on proportions of

Stratification

Stratification is another commonly used method to adjust for systematic differences between the treatment and control groups in a study.4 An example of this is found in an observational study published by Thuny et al.,9 comparing whether the timing of surgery (≤ 1 week vs > 1 week) after the beginning of antibiotic therapy influences mortality and morbidity in adults with complicated infective endocarditis. To address differences in baseline characteristics, propensity scores were generated for

Covariate Adjustment

The propensity score can also be used in a regression (covariance) adjustment.2 An example of this is a study investigating whether metabolic syndrome (MetS) was associated with late mortality in patients undergoing coronary artery bypass grafting (CABG).28 To adjust for differences in baseline covariates, the authors used propensity-score modelling, with the dependent variable being MetS and multiple covariates including sex and diabetes. The investigators elected to estimate adjusted hazard

Inverse Probability of Treatment Weighting

Another means of using the propensity score is to create a weight based on the score itself. An example of this was demonstrated in a recent study of nearly 190,000 patients by Weintraub et al.30 comparing percutaneous coronary intervention (PCI) with CABG using the American College of Cardiology Foundation and the Society of Thoracic Surgeons databases. As expected, many of the key baseline covariates were significantly different between the 2 groups; the authors addressed this by using

Discussion

Observational studies of treatment effects have inherent limitations, including potential imbalances of known and unknown confounders between treatment and control groups. The propensity score is a balancing score that makes the distribution of measured covariates similar between the treatment and control groups.4 Once propensity scores are estimated and 1 of the methods described here is applied, appropriate balance diagnostics should be performed. Presence of imbalances mean that the

Conclusions

In summary, using propensity scores is a good technique in observational studies to help achieve a better balance between the treatment and control groups. Of the 4 methods using propensity scores, matching and IPTW seem to perform better in reducing bias than do stratification and covariate adjustment. The matching technique would be ideal in an observational study in which there are more patients in the control group than in the treatment group and the resulting match is well balanced; in

Funding Sources

S.D. is supported by the Vanier Canada Graduate Scholarship, Canadian Institutes of Health Research. P.C.A. is supported by a Career Investigator award from the Heart & Stroke Foundation of Canada. J.V.T. is supported by a Canada Research Chair in Health Services Research and a Career Investigator Award from the Heart and Stroke Foundation of Canada. D.T.K. is supported by a Clinician Investigator Award from the Heart and Stroke Foundation of Canada. C.D.M. is supported by a merit award from

Disclosures

The authors have no conflicts of interest to disclose.

References (32)

  • E. Angeloni et al.

    Metabolic syndrome affects midterm outcome after coronary artery bypass grafting

    Ann Thorac Surg

    (2012)
  • B. Sibbald et al.

    Understanding controlled trials. Why are randomised controlled trials important?

    BMJ

    (1998)
  • R.B. D'Agostino

    Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group

    Stat Med

    (1998)
  • G. Heinze et al.

    An overview of the objectives of and the approaches to propensity score analyses

    Eur Heart J

    (2011)
  • R.B. D’Agostino

    Propensity scores in cardiovascular research

    Circulation

    (2007)
  • P.R. Rosenbaum et al.

    The central role of the propensity score in observational studies for causal effects

    Biometrika

    (1983)
  • P.C. Austin

    A critical appraisal of propensity-score matching in the medical literature between 1996 and 2003

    Stat Med

    (2008)
  • M.T. James et al.

    Renal outcomes associated with invasive vs conservative management of acute coronary syndrome: propensity matched cohort study

    BMJ

    (2013)
  • J.V. Tu et al.

    Effectiveness and safety of drug-eluting stents in Ontario

    N Engl J Med

    (2007)
  • F. Thuny et al.

    The timing of surgery influences mortality and morbidity in adults with severe complicated infective endocarditis: a propensity analysis

    Eur Heart J

    (2011)
  • P.C. Austin

    Propensity-score matching in the cardiovascular surgery literature from 2004 to 2006: a systematic review and suggestions for improvement

    J Thorac Cardiovasc Surg

    (2007)
  • P.C. Austin

    Primer on statistical interpretation or methods. report card on propensity-score matching in the cardiology literature from 2004 to 2006: a systematic review

    Circ Cardiovasc Qual Outcomes

    (2008)
  • M.M. Joffe et al.

    Invited commentary: propensity scores

    Am J Epidemiol

    (1999)
  • P.C. Austin

    An introduction to propensity score methods for reducing the effects of confounding in observational studies

    Multivariate Behav Res

    (2011)
  • P.C. Austin

    A tutorial and case study in propensity score analysis: an application to estimating the effect of in-hospital smoking cessation counseling on mortality

    Multivariate Behav Res

    (2011)
  • P.C. Austin et al.

    A comparison of the ability of different propensity score models to balance measured variables between treated and untreated participants: a Monte Carlo study

    Stat Med

    (2007)
  • Cited by (207)

    View all citing articles on Scopus

    See page 265 for disclosure information.

    View full text