The appropriateness method has acceptable reliability and validity for assessing overuse and underuse of surgical procedures

Objective: To summarize the findings of methodological studies on the RAND/University of California Los Angeles (RAND/UCLA) appropriateness method, which was developed to assess if variation in the use of surgical procedures is because of overuse and/or underuse.

Study design and setting: A MEDLINE literature search was performed. Studies were included if they assessed the reliability or validity of the RAND/UCLA appropriateness method for a surgical procedure or the effect of altering panelist composition or eliminating in-person discussion between rating rounds. Information was abstracted on procedure, study design, and findings.

Results: One thousand six hundred one titles were identified, and 37 met the inclusion criteria. The test-retest reliability is good to very good (kappa, 0.64-0.81) for total knee and hip joint replacement, coronary artery bypass grafting (CABG), and carotid endarterectomy (CEA). The interpanel reliability is moderate to very good (kappa, 0.52-0.83) for CABG and hysterectomy. Construct validity has been demonstrated by comparing the appropriateness method with guidelines and/or evidence-based approaches for endoscopy, colonoscopy, CABG, hysterectomy, and CEA. Predictive validity has been studied for cardiac revascularization, in which concordance with appropriateness classification is associated with better clinical outcomes.

Conclusion: Our findings support use of the appropriateness method to assess variation in the rates of the procedures studied by identifying overuse and underuse. Further methodological research should be conducted as appropriateness criteria are developed and implemented for a broader range of procedures.