Table 1

Characteristics of the studies included (n=20 studies)

Study	Perspective of quality of care	Level of care	Proposed method of QI derivation	Evidence synthesis	Consensus method	Testing/implementation
Asch et al36	NR	Process	Medical records	Literature review, not specified to be systematic	RAND approach/modified Delphi method	Implemented in 12 Veterans Health Administration care systems and 12 US communities. Average reliability: presence of a condition (κ=0.83), indicator eligibility (κ=0.76) and indicator scoring (κ=0.80)
Barber et al34	Healthcare professionals, organisational and patients	Structure, process and outcome	NR	Integrative review including an update of an earlier conducted systematic review	Modified RAND-UCLA appropriateness method	Feasibility assessed by the expert panel during the Delphi rounds
Blackburn et al35	Patients	Process and outcome	Patient-reported questionnaire	Used an earlier published systematic review21	Four discussion groups with the research team	Not tested/implemented
Broadbent et al48	NR	Process	Medical records	Used QIs of a published study,6 and of the National Primary Care Research and Development Centre	NR	QIs implemented in 18 general practices in the UK
Doubova and Perez-Cuevas37	Healthcare professional	Process	Electronic health records	Literature review, not specified to be systematic	Modified RAND/UCLA appropriateness method	QIs implemented cross-sectional in four family medicine clinics in Mexico City
Grypdonck et al49	Healthcare professional	Process	NR	Literature review, not specified to be systematic	RAND-modified Delphi method	Not tested/implemented
Hardcastle et al38	Healthcare professional	Process	Patient interview surveys	Used QIs of an earlier published study40	Modified RAND/UCLA appropriateness method	Feasibility of the survey use assessed by an expert panel of clinicians
Jansen et al43	Researchers	Process and outcome	Physiotherapist self-reported recording forms	QIs were derived from the Dutch physiotherapy guideline on KHOA	Independent assessment of QIs by two authors	QIs implemented by 27 physical therapists who recorded patient and treatment characteristics of at least five consecutive patients
MacLean31 : ACOVE-1 (supporting article: Shekelle et al)26	NR	Process	Medical records, administrative data and patient or proxy interview	Systematic review	Modified RAND/UCLA appropriateness method	Not tested/implemented
MacLean et al33 : ACOVE-2 (supporting article: Pencharz and MacLean)29	NR	Process	Medical records, administrative data and/or patient or proxy interview	Systematic review	Modified RAND/UCLA appropriateness method	Not tested/implemented
MacLean et al: ACOVE-332	Healthcare professionals	Process	Medical records, administrative data and/or patient or proxy interview	Systematic review	Modified RAND/UCLA appropriateness method	Not tested/implemented
Marshall et al19	Healthcare professionals	Process	Electronic and paper records from the general practice	Literature review, not specified to be systematic	RAND/UCLA appropriateness method	Field-testing on 1600 randomly selected patient records in 16 general practices
Moore39	Healthcare professionals	Process	Medical records	Literature review, not specified to be systematic	RAND/UCLA appropriateness method	Feasibility of the QIs assessed by the expert panel
Østerås et al46 (supporting article: Østerås et al30	Researchers	Process	Patient self-reported questionnaire	Literature review, not specified to be systematic	Critical judgement by researchers and national and international colleagues that used the questionnaire in different settings	Feasibility of QIs assessed during 2010–2014 in a Norwegian OA cohort (V1). Revised version, the OA-QI v2, was then pilot-tested by 11 of the members in the Patient Research Partner Panel at Diakonhjemmet Hospital. Test–retest к=0.38–0.85, exact agreement from 69% to 92%. The ICC for all 16 items was 0.89
Peter et al44	Healthcare providers	Process	PT self-reported online questionnaire	QIs derived from the Dutch physiotherapy guideline on KHOA	Rating of recommendations of guideline by an expert panel of PTs	Pilot-tested by 15 PTs and three experts. Consecutively, pilot test is done among expert (n=51) and PTs (n=192) in the Netherlands. Test–retest reliability: ICC=0.89
Saliba et al42	NR	Process	Not reported	Adapted ACOVE-1 set for nursing homes	Modified Delphi process	Not tested/implemented
Smith et al41	NR	Process	Not reported	Adapted ACOVE-1 set for home-based care	Modified Delphi process	Not tested/implemented
Steel et al40	NR	Process	Medical records	Adapted ACOVE-1 set for UK healthcare system	Modified RAND/UCLA appropriateness method	Not tested/implemented
Vandenberghe et al47	NR	Process	Paper registration sheets or electronic patient records	NR	NR	QIs implemented cross-sectionally in the general practices in Belgium and compared between a pooled and restricted database
Wierenga et al45	NR	Process	Medical records and a hospital information system	Adapted ACOVE-1 set for in-hospital pharmaceutical care	Expert panel review methods	Feasibility assessment with ten elderly patients. Reliability tested with ten randomly selected patients; к=0.88 (95% CI 0.75 to 1.00); ICC=0.80 (95% CI 0.63 to 0.90)

More information on the data extraction and quality assessment for each study is provided in online supplemental file 3.
ACOVE, Assessing Care of Vulnerable Elders; ICC, intraclass correlation coefficient; KHOA, knee and hip osteoarthritis; NR, not reported; OA, osteoarthritis; PT, physiotherapist; RA, rheumatoid arthritis.