Article Text
Abstract
Objectives To evaluate the quality of published multiple treatment comparison (MTC) meta-analyses on biologic disease modifying antirheumatic drugs (bDMARDs) for rheumatoid arthritis (RA), and to identify methodological issues that can explain the discrepancies in the findings of these MTCs.
Methods We searched MEDLINE for MTCs of bDMARDs for RA. Following the PRISMA guidelines, we extracted a large set of methodological items. These comprised of inclusion/exclusion criteria, information sources, reported results and outcomes measures, approaches to dealing with differing response profiles to available treatments, monotherapies versus combination therapies, and potential sources of heterogeneity.
Results We identified 13 published MTCs, of which nine were published since 2009. Despite similar stated eligibility criteria and objectives across MTCs, we identified major discrepancies in the estimated treatment effects, the inclusion of trials and analytic approaches. The number of included trials was typically much smaller than the number of eligible trials at the time of publication. Three out of six MTCs including patients of differing response profiles inappropriately combined DMARD-naive and DMARD-inadequate responder patients in the analyses. Four out of eight MTCs that considered both monotherapy and combination therapy (ie, concomitant DMARD) did not adjust for the potential effect modification. Half of the identified MTCs did not explore potential sources of heterogeneity, and the explored sources varied considerably. Last, most MTCs only included one or two efficacy outcomes (eg, ACR50) and only two considered health related quality of life outcomes (eg, HAQ).
Conclusions The identified methodological shortcomings and inconsistencies most likely explain the observed discrepancies in findings across MTCs.
- Anti-TNF
- Epidemiology
- DMARDs (biologic)
- Outcomes research
Statistics from Altmetric.com
Background
Over the last decade, a number of biologic disease modifying antirheumatic drugs (bDMARDs) have been approved for the treatment of established rheumatoid arthritis (RA). Currently, nine bDMARDS are available (abatacept, adalimumab, anakinra, certolizumab, etanercept, golimumab, infliximab, rituximab and tocilizumab) and there are a number of forthcoming drugs that are likely to receive regulatory approval. These bDARMDs are typically used when patients fail to respond to traditional disease modifying antirheumatic drugs (DMARDs), such as methotrexate (MTX).
Numerous randomised clinical trials (RCTs) have established the effectiveness of each of the bDMARDS individually,1 but the comparative effectiveness between the bDMARDS is still an area of contention. When synthesising data from published RCTs on several treatments, one should employ methods that allow establishing comparative effectiveness between all treatments in a statistically robust manner. Indirect and multiple treatment comparison (MTC) meta-analysis is a relatively new but advanced extension of conventional meta-analysis that allows just that.2 ,3 The continual introduction of novel bDMARDs has resulted in a large number of published MTCs.4–16 There are now more MTCs of bDMARDS for the treatment of RA than there are MTCs in any other disease field,17 and many of these arrive at different conclusions about the comparative effectiveness of the available bDMARDs. Such discrepancies make it difficult for clinicians and policy makers to determine effectiveness and reimbursement schemes for bDMARDs. This is particularly the case when MTCs are used as part of health technology assessment (HTA) submissions.
RA is a complex area with many clinical factors potentially influencing the relative efficacy and safety of the individual bDMARDs (eg, treatment and response history, concomitant use of MTX, dosing of the administered bDMARD, and pretreatment disease duration).8 ,10 ,11 ,15 ,16 ,18 Similarly, MTC is a relatively new statistical approach that has little established guidance on how to perform a gold-standard analysis.17 As a result, the expertise in conducting MTCs is limited. Yet, to ensure valid and generalisable findings, rigour is required in many aspects of the analysis from inclusion criteria through to interpretation.19–23 While HTA submissions typically report on more outcomes than a journal publication, the principle of completeness for RA MTCs should be similar.
In this article, our aim was to evaluate the quality of published MTCs on bDMARDs for RA. We also aimed to identify methodological issues that can explain the substantial discrepancies in the findings of these MTCs. We make recommendations as to what components are necessary and important in MTCs of bDMARDs for RA. This paper will inform authors of future MTCs, and thus hopefully ensure consistency in findings among future publications. In particular, we propose criteria for producing high quality MTCs of bDMARDs for RA as part of an HTA submission.
Methods
Eligibility criteria
We included any systematic review of bDMARDs used for RA that applied an indirect comparison or MTC and measured the clinical efficacy of these drugs. We only included published articles in refereed journals. We did not include systematic reviews of more than one biologic in which the effectiveness of each included biologic was only obtained separately through pairwise meta-analysis, and we did not include conference abstracts as data are frequently incomplete.
Search strategy
We searched the following databases from inception to week 18 (April 30–May 6 2012): MEDLINE, EMBASE and the Cochrane Library. We recursively searched the reference lists of all identified eligible articles.
Data extraction
Following the recommendations of the preferred reporting items for systematic reviews and meta-analysis (PRISMA) statement,24 we extracted data on the stated objectives and methods of the retrieved MTC publications: study eligibility criteria, information sources (ie, databases searched), data items, bias risk assessment, effect measures, statistical methods and additional analysis. For the statistical and additional analyses, we particularly extracted information on the underlying methods/models employed, statistical treatment of DMARD-naive versus DMARD-experienced patients, statistical treatment of concomitant use of MTX/DMARDs, heterogeneity analysis and other sensitivity analyses. We also extracted the corresponding items for the results as well as the treatment effects estimates.
Analysis
Comparison of stated objectives
We compared the stated objectives across the identified MTCs and whether these objectives matched the methods within each of the individual MTC publications.
Comparison of characteristics of search strategy and eligibility criteria
We compared the information sources and the number of trials included across MTC reviews. We compared the eligibility criteria used across the identified published MTCs. In particular, we compared the eligible populations, the types of eligible trials, the minimum duration of therapy and the considered interventions.
Comparison of study quality (bias risk) assessments
We evaluated whether the authors of the identified MTC publications had assessed the quality or bias risk of the included studies, and if so, the tools they had employed to do so (eg, GRADE,25 Cochrane bias risk tool26 or the Jadad scale27).
Comparison of completeness of trial inclusion
We assessed the completeness of each MTCs trial inclusion by compiling a list of trials identified across all reviews and comparing this with the set of trials included in each MTC. In addition, we assessed how well each of the considered interventions (bDMARDs) were represented in the identified MTCs by plotting horizontal bar plots for the number of trials investigating the respective interventions in each review.
Comparison of employed statistical analyses
For the comparison of statistical analyses we considered several items. First, we compared the chosen outcome measures of efficacy. Second, we compared whether and how the MTCs dealt with the analyses of DMARD-naive versus DMARD-experienced patients. Third, we compared the employed method/model for obtaining pooled estimates of efficacy (eg, adjusted indirect comparison, frequentist regression or Bayesian MTC). Fourth, for reviews that included monotherapy and combination therapy (bDMARDs with concomitant MTX/DMARDs), we compared how these two types of interventions were distinguished and handled in the statistical analysis. Fifth, we compared the types and amount of analyses conducted to explore potential sources of heterogeneity. And last, we compared the employed sensitivity analyses (eg, exclusion of important ‘odd’ trial results).
Comparison of chosen outcomes and efficacy estimates
We compared which outcomes had been chosen to measure efficacy. We compared reported estimates of efficacy and associated statistical significance across reviews. For each review, we ordered the estimated effect measures by magnitude of effect for each of the considered outcomes. This allows insight into the inconsistency of reported effect estimates and reported treatment rankings across reviews. We made these comparisons in connection with the number of bDMARDs that were available (ie, at least one published trial) at the time the MTCs were published.
Results
We identified 13 publications using adjusted indirect comparison or MTC statistical techniques to compare the efficacy of bDMARDs for the treatment of RA.4–16
Comparison of objectives
Table 1 shows the stated objectives. No considerable discrepancies were observed across MTCs. In general, all reviews aimed to estimate the comparative effectiveness of bDMARDs for the treatment of active RA. Patients were often vaguely specified in the objectives (eg, ‘patients with active RA’), and outcomes were never specified.
Comparison of characteristics of search strategy and eligibility criteria
Table 1 shows the characteristics of the search strategy and eligibility criteria of the identified MTC journal publications. Similarly, online supplementary table S1 shows the compiled list of trials included in the identified MTC publications. Most MTCs searched MEDLINE, EMBASE and the Cochrane library of registered trials, used 6-month follow-up (even reviews that only required 3 months still used 6 months as the primary outcome), and considered most of the American College of Rheumatology (ACR) measures for efficacy. Eight of the identified MTCs included both monotherapy and combination therapy trials, whereas five included only combination therapy trials. All MTCs considered DMARD-inadequate responders (IR), but some additionally considered both DMARD-naive patients and anti-tumour necrosis factor (TNF)-IR patients.
Comparison of study quality (bias risk) assessments
Table 1 also shows the study quality (risk of bias) assessment tools that were employed throughout the identified MTC publications. Only five of the 13 MTCs had assessed study quality. Of these, three used the Jadad scale,27 one used the AMSTAR scale (since this review compiled data from six Cochrane pairwise systematic reviews)28 and one used the US Preventive Services Task Force and the National Health Service Center for Reviews and Dissemination Scales. In general, the five MTCs that evaluated the study quality seemed to infer that the included trials were of sufficient quality (low risk of bias) to warrant pooling.
Comparison of completeness of trial inclusion
By 2011, the total number of trials for possible inclusion among all identified MTC publications was 61. Among these, seven trials included only DMARD-naive patients, 46 trials included only DMARD/MTX-IR patients, four trials included a mix of the two and four trials included only anti-TNF-IR patients. Figure 1 shows the cumulative number of trials published each year from 1997 to 2011, and thus, the maximum number of eligible trials each year. By the end of 2008, there were already 38 published trials on DMARD-IR patients, an additional six on DMARD-naive patients and an additional three on a mix of the two. However, with the exception of one MTC, all MTCs published between 2009 and 2011 included less than 31 trials, and the median number of trials included was 18. Online supplementary table S1 shows that the published MTCs have generally included considerably different sets of trials. Figure 2 displays the representations of interventions (compared with control) in the identified MTCs. Overall, the representation of each intervention varied considerably across MTCs. The representation of older anti-TNF inhibitors (adalimumab, etanercept and infliximab) varied considerably over time, and these are not necessarily better represented in later MTCs. The newer biologics (abatacept, certolizumab, golimumab, rituximab and tocilizumab) were, for reasons of publication date (see figure 3), only present in all or some of the later MTCs, but their representation was still variable. Anakinra was less likely to be included in later MTCs.
Figure 4 presents the largest possible treatment network including trials on DMARD/MTX-IR patients only (a total of 46 trials). This network graphically reduces to a ‘star’ network (a network where all interventions are only connected to the control) if monotherapy and combination therapy are lumped.
Comparison of employed statistical analyses
Table 2 provides an overview of the employed statistical methods throughout the identified MTCs. Three of the four MTCs published up till 2008 used adjusted frequentist indirect treatment comparison meta-analysis and one used Bayesian MTC. Among the nine MTCs published in 2009 or later, six used Bayesian MTC meta-analysis, two used a frequentist MTC meta-analysis and one used adjusted frequentist indirect treatment comparison meta-analysis.
Six MTC publications included DMARD-naive patients, anti-TNF-IR patients or both in addition to DMARD-IR patients. Of these, two MTCs produced separate/stratified analysis by patient groups, one produced an interaction analysis including the group term in a regression model, and three lumped data across groups without controlling or separating (two lumped DMARD-naive and DMARD-IR, one lumped DMARD-IR and anti-TNF-IR).
Among the eight MTC publications that included both monotherapy and combination therapy, only one published before 2011 (out of four MTCs) accounted for the potential effect modification associated with concomitant use of MTX–DMARDs. One of the reviews that did not control for this potential effect modification, however, performed a meta-regression interaction analysis to assess the effect of concomitant MTX/DMARD use. Among those published in 2011, all initially adjusted for concomitant use of MTX/DMARD, one did not report controlled treatment effects as this covariate was not statistically significant in the meta-regression model.
Eight of the 13 identified MTCs explored potential sources of heterogeneity: six used meta-regression and two used sensitivity analysis (by exclusion of some trials). However, the number and types of covariates varied across MTCs. All covariates considered across MTCs were dose of biologics (all, recommended, low, high), concomitant use of MTX/DMARDs, mean baseline duration of RA, type(s) of drugs previously failed, whether anti-TNFs were previously failed, whether the bDMARD intervention was an anti-TNF agent, timeframe (<6 months, 6–12 months, >12 months), baseline C reactive protein, Health Assessment Questionnaire (HAQ) baseline score, baseline swollen joint count and tender joint count.
Four of the 13 MTCs included sensitivity analyses that comprised of exclusion of single trials. Table 3 presents the excluded trials and stated reasons for exclusion in the four MTCs. Three of these four MTCs excluded the TEMPO trial, but each MTC stated different reasons for exclusion of this trial. In addition, one MTC excluded the Combe 2006 trial because sulphasalazine, and not MTX, was the administered background DMARD; one excluded the GO-FORWARD trial, and separately the RAPID-1 and RAPID-2 trials, since these use a rescue regimen; and one MTC excluded the SERENE and LITHE trials as these were only available as conference abstracts at the time.
Comparison of chosen outcomes efficacy estimates
Table 1 shows the outcomes chosen to measure efficacy across the identified MTCs. ACR outcomes were used in all MTCs, whereas HAQ was only used in two and DAS was only used in one. ACR50 was used in all 13 MTCs, ACR20 was used in eight of the 13 MTCs and ACR70 was used in five of the 13 MTCs. Table 4 shows the bDMARD treatments ordered (ranked naively) by estimated magnitude of ACR effect in each of the identified MTCs. Online supplementary table S2 shows the same information, but additionally includes the pairwise comparative effect estimates reported in the publications. All biologics were consistently shown to be superior to placebo, and anakinra was consistently the least effective of biologics. Several fluctuations were observed for the estimated treatment effects and in the naive treatment rankings across MTC publications. In most instances, the data of the included trials were insufficient to demonstrate any statistical significance in treatment effects between biologics. However, certain significant findings did occur, but these were contradictory with the findings of other MTCs. One highly discrepant example can be found between Turkstra et al14 and Guyot et al.15 In their MTC, Turkstra et al14 estimated that the effect of certolizumab (compared with control) for ACR50 was associated with an OR of 22.3 (95% Credible Intervals (CrI) 9.93 to 43.1), whereas adalimumab (compared with control) was associated with an OR of 3.34 (95% CrI 1.98 to 5.39), thus, a statistically significant difference. However, Guyot et al15 estimated that certolizumab (compared with abatacept) for ACR50 was associated with an OR of 0.35 (95% CrI 0.08 to 1.35), whereas adalimumab (compared with abatacept) was associated with an OR of 0.40 (95% CrI 0.09 to 1.40), thus, two highly similar estimates between these two biologics. Another highly discrepant example can be found between Devine et al10 and Salliot et al.12 In their MTC, Devine et al10 estimated that tocilizumab (compared with control) was associated with a log OR of 1.67 (95% CrI 1.31 to 2.07) and that rituximab was associated with a log OR of 1.61 (95% CrI 0.55 to 2.2.85), thus, two quite similar estimates. However, Salliot et al12 estimated that tocilizumab (compared with control) was associated with an OR of 12.32 (95% CrI 5.11 to 29.7), and that rituximab was associated with an OR of 3.12 (95% CrI 2.07 to 4.71), thus, a statistically significant difference.
Discussion
The identified inconsistencies among RA MTCs have implications and utility for future evidence synthesis research in RA. The treatment of RA is one of the largest budget considerations for both patients and payers. The inconsistencies between MTCs, particularly in the context of HTAs, are concerning given their likely impact on health economic evaluations and drug approvals. Given the importance of accurate analyses to inform clinical and economic decision-making, we make the following recommendations for the future conduct of MTCs. Our recommendations are intended for MTCs used in HTAs as well as MTCs intended for journal publication. While the principles are the same, subtle differences seem sensible due to journal word limit and underlying intent. In table 5, we present our recommendations for RA MTCs produced in the context of HTAs and journal publications respectively.
Identifying robust objectives
The considerable vagueness of the stated objectives offers a possible explanation for the methodological inconsistencies observed across published RA MTCs. Without clearly stated objectives following the traditional PICO format (Patients-Intervention(s)-Comparator(s)-Outcomes), the potential to err or deviate from protocol is great.29 We provide clear recommendations for putting together robust objective(s) in table 5.
Inclusion of trials
The substantial inconsistencies in the included trials across the published MTCs (table 1, online supplementary table S1 and figure 2) offer a plausible explanation for the fluctuations in magnitudes of treatment effects and (naive) treatment rankings that were observed across the reviewed MTCs (table 4 and online supplementary table S2). Differences in the inclusion/exclusion criteria and database search algorithms used for each of the individual MTCs are likely reasons for these inconsistencies in trial inclusion. In the light of these observed inconsistencies it seems reasonable to insist that future efforts to synthesise results from RCTs of bDMARDs for RA should at least consider searching all sources used in the published MTCs, as well as review the compiled list of trials (see online supplementary table S1) from these reviews for eligibility).
Given the broader range of questions that HTAs can answer due to usually unlimited word counts, it seems reasonable to require that HTAs especially consider a wider range of trials and incorporate the necessary analytical measures (eg, heterogeneity analysis). However, there are situations where restriction may be necessary. One example is the inappropriate combination of MTX/DMARD-naive and -experienced patients, as discussed below. A drug agency specific example would be that of National Institute for Health and Clinical Excellence, which currently (August 2012) only considers MTCs where RA patients have had an inadequate response to at least two DMARDs, and thus, this criterion must be part of the HTA trial eligibility criteria.
Statistical analysis (1): dealing with DMARD-naive and DMARD-experienced patients
A key issue in baseline differences is the patient's experience with previous therapy. Patients who are MTX/DMARD-naive will, on average, have vastly different response profiles with MTX/DMARD than patients who are DMARD-experienced. In DMARD-naive patients, high doses of MTX, for example, will likely yield responses close to that of biologics and the combination will be somewhat better (as seen in the TEMPO trial). In DMARD-experienced patients (the template for regulatory drug approval studies), the MTX/DMARD group can only be regarded as close to placebo, and the biologics group will do much better, regardless of whether MTX/DMARD is continued (combination therapy) or not (monotherapy). Thus, it seems reasonable to avoid any lumping of MTX/DMARD-naive and MTX/DMARD-experienced patients. Likewise, patients who have previously failed a bDMARD (currently only studied for anti-TNF-IR patients) should not be pooled with those who are bDMARD-naive.
Statistical analyses (2): combining monotherapy and combination therapy trials
The potential effect modification by concomitant use of DMARDs/MTX and the issue of how to combine monotherapy and combination therapy trials is another challenging aspect in RA research synthesis. All MTCs published in 2011 that included both monotherapy and combination therapy trials investigated the effect of concomitant use of DMARDs/MTX using meta-regression. However, DMARDs/MTX did not consistently yield a significant effect modification across MTCs, and one review chose to leave out this covariate in the primary analysis based on lack of statistical significance. Since future RA MTCs (including those of patients with bDMARD inadequate response) will undoubtedly continue to include both monotherapy and combination therapy trials, it is important to establish whether the concomitant use of DMARDs/MTX does in fact yield an effect modification on the relative comparative treatment effects. This will require more refined analyses than what have currently been performed among the MTCs.
Alternative and potentially better approaches are available, each of which recognise that the effect modification by concomitant use of MTX/DMARDs may both be the same in combination with all bDMARDs. First, one can analyse monotherapy trials and combination trials separately (ie, in two separate treatment networks). Second, one can regard monotherapy and combination therapy as different, but still investigate their comparative effectiveness by combining all treatments in one network (see figure 4). Last, one can extend the meta-regression model to allow for differing effect modifications associated with MTX/DMARD used across bDMARDs.
Combining all trials as if the relative efficacy of bDMARDs is the same regardless of concomitant use of MTX/DMARD should be avoided. At least one of the above three approaches should be considered. Clear rationale should be provided for why the chosen approach is most suitable for the data at hand. Last, choosing a second of the above three approaches as sensitivity analysis to establish robustness of findings is advisable. Especially in HTAs, where word count is not an issue, such a sensitivity analysis would be justified.
Statistical analysis (3): exploring heterogeneity and effect modification by clinical covariates
Several potential sources of heterogeneity were explored among published MTCs using subgroup analysis, meta-regression and sensitivity analyses. Many of the trial level covariates such as baseline HAQ score and baseline DAS score are correlated, and so, may not all need to be considered. Other trial level covariates such as treatment duration may only be necessary to include in specific analytical situations. For example, most MTCs pooled 24 weeks/6 months data where the timeframe restrictions were tight (eg, ±4 weeks). In this case, including treatment duration would not be necessary, whereas it may be for wider spans of included treatment durations.
As with concomitant use of MTX/DMARDs, some trial level covariates may be statistically significant depending on the number and set of trials included for the analysis. No consensus has been reached as to which trial level covariates are particularly important to consider. Thus, it seems reasonable to require that future MTCs continue to include covariates in meta-regression analysis that may seem important to the particular MTC.
Extending the discussion of inclusion of trials, HTAs in particular have more space for detailed analyses (both a priori and post hoc exploratory). We put forward that MTCs in HTAs can readily explore effect modification by a number of identified variables (refer to table 2).
Statistical analysis (4): sensitivity analysis by exclusion of trials
A number of trials were excluded in sensitivity analyses for a variety of reasons in four of the 13 MTCs. However, eight of the 13 MTCs did not perform any sensitivity analyses by trial exclusion. In RA MTCs, the line between trial inclusion and exclusion may be fine. In situations where the inclusion of one or more trials is not an easy decision, it will be important to consider whether these trials jeopardise the internal validity. Sensitivity analyses by exclusion of such trials may likely be warranted in this case. Sensitivity analyses may also be warranted for other trial design reasons.
The TEMPO trial was excluded in sensitivity analyses in three identified MTCs. Although three different reasons were stated, the main rationale for excluding the TEMPO trial is essentially that the population is not entirely MTX-IR patients. It is possible that other MTCs also debated whether to include or exclude this trial. The RAPID-1, RAPID-2 and GO-FORWARD trials were all excluded because the trial design incorporated termination of treatment in patients where the treatment did not show efficacy. Again, however, other MTC chose to include these trials. In addition, other trials that have not yet been subjected to sensitivity analysis by exclusion in the MTCs may present important limitations. Considering the limitations of the included trials is an important part of confirming robustness of findings in MTCs, and authors of future MTCs of bDMARDs in RA are well advised to learn from previous published MTCs as well as HTAs.
As with heterogeneity analysis of clinical covariates, sensitivity analysis by trial exclusion is an area where HTAs related MTCs could readily go into more detail than MTCs in journal publications. This may be particularly relevant for the area of RA where contention seem to exist about a non-negligible number of published RCTs. When sensitivity analyses seem necessary, the rationale should be stated clearly. For example, with the TEMPO trial the differing characteristics should be stated clearly.
Statistical analysis (5): choice of efficacy measure
Several efficacy measures have been used across RA RCTs, and so, considering a large body of these measures can paint a more detailed and robust picture of comparative effectiveness. Yet, less than half of the 13 MTCs looked at all three ACR outcomes, and only two MTCs considered health related quality of life measures (ie, HAQ and DAS). Since methodological limitations such as bias and power should be assessed separately for each outcome, it is understandable that MTCs published in peer-reviewed journals have only focused on a selection. However, as patients responding at ACR20, ACR50 and ACR70 are partially the same, one should consider analysing these three responses together in one statistical model, not with three separate statistical models. For HAQ, DAS and other health related quality of life measures (eg, the EULAR score), an MTC taking into account all of the above points is needed. As for HTAs, no word count restrictions apply, and thus, all relevant measures should be considered in the analysis. Further, health related quality of life measures naturally feed into economic evaluations like cost-effectiveness analysis, and so, seem highly relevant to include in HTAs.
Concluding remarks
Comparative effectiveness of bDMARD for the treatment of RA is an area of great contention. The evidence base of RCTs on the topic lends itself to evaluation using MTCs. However, the clinical and statistical complexity of such analyses makes it particularly difficult to achieve methodological consistency and congruent statistical inferences across published MTCs. This is particularly problematic when MTCs are used as part of HTA to determine the best reimbursement schemes.
Our review of key methodological components in published RA MTCs demonstrates that several important inconsistencies exist. Based on these identified inconsistencies, we have provided a discussion of lessons to be learnt for future research synthesis efforts—both for HTAs and journal publication MTCs. While future MTCs and reviews of MTCs may reveal further issues to be considered (eg, whether and how to best include outcomes like HAQ and DAS), the bottom line remains clear: to reduce current contention in comparative effectiveness research of biologics for RA, efforts should be made to streamline the employed research synthesis methodology. Our overview of recurrent methodological shortcomings in published MTCs provides a first step in this direction.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online tables
Footnotes
-
Handling editor Tore K Kvien
-
Contributors KT and EM conceived the design of the study. KT drafted the first manuscript. KT and ED extracted the data. All authors contributed to the interpretation of the findings and to the writing of the final version of the manuscript.
-
Funding Pfizer UK.
-
Competing interests Kristian Thorlund and Edward Mills have consulted Merck, Pfizer, Nycomed or GlaxoSmithKline on multiple treatment comparison and systematic review issues. Kristian Thorlund and Edward Mills have received grant funding from the Canadian Institutes of Health Research (CIHR) Drug Safety & Effectiveness Network to develop methods and educational materials on MTCs. Edward Mills receives salary support from the Canadian Institutes of Health Research through a Canada Research Chair. Kristian Thorlund receives salary support from the CIHR Drug Safety & Effectiveness Network. Ping Wu and Eric Druyts are employees of Edward Mills, but did not any payments specifically for this study. Ping Wu, Eric Druyts and Antonio Avina have no conflicts of interest to report. We can think of no other relevant competing interests.
-
Provenance and peer review Not commissioned; externally peer reviewed.