Article Text
PDF
Abstract
Objectives The introduction of biological therapies for the treatment of rheumatic diseases has drawn attention to the limitations of traditional means of assessing drug safety. Consequently, a series of European academic biologics registers dedicated to this task have been established. Increasing reliance upon safety data generated from observational drug registers makes it important to convert the lessons learned from such registers into recommendations for rheumatologists embarking upon the establishment of future registers, or analysing and reporting from new and existing registers.
Methods The Task Force encompassed 11 scientists from European Rheumatology drug registers. Through an informal inventory of critical elements in the establishment of existing rheumatoid arthritis drug registers, of analytical strategies used and of limitations of their results, several ‘points to consider’—beyond established generic guidelines for observational registers/studies but with particular relevance to biologics registers on safety in rheumatology—were assembled. For each ‘point to consider’, contextual and methodological background and examples were compiled.
Results A set of seven points to consider was assembled for the establishment of new drug registers with a focus on purpose, population to be targeted, data collection, handling and storage as well as ethical and legal considerations. For analysis and reporting, nine points to consider were assembled (setting, participant, variable, statistical method, descriptive data, outcome data, main results, other analyses and limitations).
Conclusions Thoughtful design and planning before the establishment of biologics registers will increase their sustainability, versatility and raw data quality. Harmonisation of analyses and reporting from such registers will improve interpretation of drug safety studies.
Statistics from Altmetric.com
Introduction
Increasing development and clinical use of targeted biological therapies warrants careful postmarketing pharmacovigilance given the uncertain safety profiles of these ‘first in class’ compounds, for which even ‘class effects’ may be uncertain. Industry, regulators and the rheumatology community have recognised the need for observational studies to monitor the safety of these treatments. Academia-initiated biologics registers in rheumatology have been established in several European countries.1 Perhaps contrary to expectations, a number of studies addressing similar questions have (at first glance) led to less certainty rather than increased clarity. For example, estimates of the risk of serious infection attributable to anti-tumour necrosis factor (anti-TNF) therapy range from no increased risk to a more than twofold increase in risk.2,–,4 Replication of results in a second cohort is standard practice in genetic studies5 and should serve as a good model for biologics registers in rheumatology. However, without clear reporting of study design and methodological technique, it becomes impossible to understand whether discrepant results are due to chance or to inherent differences between studies.
General and generic recommendations for the conduct and reporting of clinical research, including the Consolidated Standards of Reporting Trials statement for randomised controlled trials,6 the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting observational studies7 and the Agency for Healthcare Research and Quality guide to registries evaluating patient outcomes,8 provide a good starting point for researchers who wish to establish a register or report scientific analysis of observational data. The experience of the past decade with biologics registers in rheumatology has, however, highlighted additional issues specific to safety studies of biological therapies. Against this background, the European League Against Rheumatism (EULAR) initiated a Task Force aimed at developing recommendations for rheumatologists regarding the establishment, analysis and presentation of data from biologics registers in rheumatology. Thus, rather than finding the level of scientific evidence in the literature, for example, the aim of the Task Force was to share experiences in an educative manner.
Methods
The Task Force comprised two convenors and nine scientists with experience from European drug registers in rheumatoid arthritis (RA). First, through an inventory of critical elements in the establishment of existing registers, the analytical strategies used and the limitations of their results, a preliminary list of issues of specific relevance to biologics registers in rheumatology was compiled by the two convenors (WGD and JA) and circulated to all Task Force members for additions and comments. At a subsequent meeting, this preliminary list was refined and structured by informal discussion. A preliminary report was then compiled by the convenors and circulated, with each Task Force member having responsibility for one or more sections. The final report, revised by the convenors, was re-circulated and edited by all members.
While biologics registers have been vital in understanding both safety and effectiveness, the remit of this Task Force was to consider safety. The results may, however, be of relevance also for registers/studies of effectiveness. Similarly, the results of this Task Force describe our experience from European biologics registers in RA, some of which may be applicable to other medical fields, classes of drugs or geographical regions. Acknowledging heterogeneity in the setting of, and resources for, the establishment and conduct of safety registers and practical limitations of data collection, the issues dealt with by this Task Force were called ‘points to consider’ rather than formal recommendations.
For the section on establishing a biologics register, while there is inevitable overlap with existing detailed generic guides,8 we have attempted to focus on items specific to biologics registers. For the section pertaining to analysis and reporting of safety data from biologics registers, we have reproduced the STROBE guidelines to emphasise generic guidelines for the reporting of observational research as a sound starting point, but appended item-specific points to consider. Where areas of contrasting methodology were identified in our registers, we have considered whether harmonised models are possible.
Results
Section A: Points to consider when establishing a biologics register
Purpose of the biologics register
Biologics registers have many potential benefits, including real-life assessment of drug safety, effectiveness, utilisation and economic consequences. Compared with most other outcomes, safety assessments require larger populations because of low background rates of serious adverse events. Investigators should be clear of their reasons for establishing a register, as these will affect the required size/duration of the register. Power calculations should be used to estimate sample sizes and duration of follow-up required to address the aims (see table 1 and file 1 in online supplement).
Points to consider when establishing a biologics register
Population to be targeted
Biologics registers consider an ‘exposed’ cohort of patients treated with one or several drugs under investigation. Without careful thought, this ‘exposed’ cohort may be very heterogeneous. It may include patients receiving the drug for different indications among whom the background rate of events may differ. Exposure may range from a single episode to life-long treatment. Patients may be starting treatment at the time of recruitment (incident users) or recruited during ongoing treatment (prevalent users) (see file 1 in online supplement).
Biologics registers on safety must include a meaningful comparator (‘unexposed cohort’) to act as a benchmark. Such a comparator group should be as similar as possible to the exposed group, aside from exposure to the drug under study. The comparator may consist of a population either not exposed to a drug, exposed to an alternative drug, or to the occurrence of the outcome in the treated population before or after treatment. Depending on which type of comparator is selected, slightly different contrasts with regard to safety will be made (comparing new starters on drug X with patients stable on another drug will detect the full effects of treatment with drug X, comparing new starters on drug X with new starters on drug Y will detect the marginal effects of drug X over drug Y) (see files 1 and 2 in online supplement). There should be marked overlap between the comparator and the exposed cohort in terms of characteristics that may influence the occurrence or detection of the outcome under study (eg, disease activity, calendar period of recruitment, age and gender). Eligibility criteria for inclusion into both the biologics-treated cohort and the comparator need to be clearly stated and easily accessible.
Patient profiles and treatment intensity may differ substantially between rheumatology centres. For example, higher comorbidity is seen in patients managed at referral centres. Treated and comparator patients should be drawn from similar sources to minimise selection bias. If real-life experience is important, recruitment should encompass not just single centres but the whole patient population. In the latter case, ‘population-based’ coverage is essential, although ‘population’ need not be national but may be regional or local.
Data items to be collected: treatment and treated condition
The drug(s) under observation should be recorded with its dosage(s), method of application and start and stop dates. The method of data capture should be sufficiently sophisticated to allow accurate capture of changing frequency and dosage without compromising data entry. For drugs given infrequently (eg, infusions), the treatment interval and/or the dates of treatment should be collected. If treatments are stopped, the reasons for discontinuation should be registered but should never be the reason for exclusion from further follow-up (see file 1 in online supplement).
Clinical decisions will influence which patients receive treatment, potentially introducing ‘channelling bias’ and confounding the association between treatment and risk of the safety outcome. This is inevitable, but has implications for data collection which need to be extensive enough to allow for some estimation (and statistical handling) of the degree of channelling bias or confounding. Biologics registers therefore need, up front, carefully to consider the potential confounders that may exist in order for that information to be collected at baseline and follow-up (see file 1 in online supplement). In general, data for each patient should include an achievable minimum core set of variables. Failed capture of core data for patients may lead to their exclusion from analyses, wasting effort in recruitment and follow-up. Data fields should be intuitive and unambiguous so misinterpretation during data entry is minimised, for example, through provision of a predefined range of answers.
Data items to be collected: outcomes
Information on safety outcomes must be collected in a robust, comparable, time-anchored and internationally agreed manner such as the Medical Dictionary for Regulatory Activities coding system for adverse events (http://www.meddramsso.com). Ideally, a clear-cut definition of each outcome should be available for the individual responsible for data entry. Care must be taken not to overload data collection with too many or overly detailed variables as this will increase the risk of errors and decrease care providers' or patients' willingness to report. Specific questions for prespecified outcomes of interest may increase reporting, although might introduce differential capture compared with other non-specified events. End points such as hospitalisation, cancer or death may sometimes be collected from sources other than the register itself. It is important to understand the validity of the outcome events—for example, through chart review, existing audits of external registers or by cross-referencing multiple sources of information (eg, patient-derived and physician-derived reports) (see file 1 in online supplement).
Since events have to be assigned to specific time-varying treatments, it is essential to collect dates of events as exactly as possible. Certain adverse events such as malignancies could generate several date options including onset of symptoms, date of hospitalisation or date of definite diagnosis. In such situations, the investigators should use a consistent and transparent approach and consider alternative definitions as sensitivity analyses. Some safety outcomes may need to be considered beyond drug discontinuation. A minimum observation time for each patient (eg, 5 or 10 years) is required to catch events with longer latency (see file 1 in online supplement).
Follow-up
It is recommended to follow up patients at predefined time points in order to gather temporally comparable information from all patients. If data are collected as part of routine care, the frequency should consider usual clinical practice and allow a degree of flexibility (see file 1 in online supplement).
Methods of follow-up should be as similar as possible between exposed and comparison cohorts, and for all subjects in each cohort (see file 1 in online supplement). Monitoring systems to identify overdue or missing data should be set up, and polite reminders of pending and/or overdue data should be considered. Strategies to maximise data entry and minimise losses to follow-up include provision of data for local, regional or national audit, financial reward systems and provision of a clinically useful disease chronicle to the clinician and/or patient.
Data collection process and data collectors
Biologics registers may be set up as independent studies and employ external data collectors (eg, study nurses) specifically employed for the register. Alternatively, the register may rely on reporting by physicians or other care providers, either prompted by mailed questionnaires or as an integrated part of routine care (see file 1 in online supplement). Some outcomes may be better reported by patients themselves.
It is challenging to incorporate data collection into busy clinical care. Methods should be considered for avoiding repeated data entry into clinical and research records as this is time-consuming, demotivating and introduces transcription errors. Electronic patient records may allow direct export of study data. Alternatively, study data entered at a visit may generate a clinical note for paper patient records (see file 1 in online supplement).
Data collection from patients can be done in the waiting room using touch screens. This has the advantage of reducing transcription with its problems of time and possible error. In addition, physicians have the benefit of summated patient-reported data during the clinical encounter. If multiple data sources (eg, patient diaries and questionnaires, physician forms and nurse forms) are used simultaneously, the type of information collected in one should be coordinated with the others to avoid ‘double counting’.
Data handling and storage: ethical and legal considerations
Existing biologics registers have highlighted many issues surrounding data ownership, data access, relation to external sponsors, feedback to reporters and compliance with national legislation and rules for protection of privacy. It was felt that, because these issues were not specific to biologic registers, they are discussed only in file 1 in the online supplement.
Section B: Points to consider when undertaking analyses and reporting results from biologics registers in rheumatology
Setting
Information on access to treatment, eligibility criteria for treatment and drug penetration within the source population should be reported. Varying drug penetration by country has implications for the comparability of studies9 (see file 2 in online supplement). Drug penetration may also vary over time within countries, so authors should factor in changing prescribing patterns over time. Similarly, if biological agents introduced at different calendar time points are to be contrasted to each other (eg, infliximab vs adalimumab) or if awareness of a safety issue may have led to changes in clinical practice over time (eg, TNF inhibitors and tuberculosis risk),10 such contextual information needs to be provided in the report (table 2).
Specific points to consider when undertaking analyses and reporting safety results from biologics registers in rheumatology (right column) in addition to general recommendations for improving the reporting of observational studies (STROBE guidelines, left column*)
Participants
The ‘exposed’ cohort may include a spectrum of drug treatment scenarios ranging from a single treatment episode to life-long treatment for chronic disease. The cleanest exposed cohort in biologics registers comprises patients starting treatment with no prior exposure to that drug. If patients have prior exposure to the drug or class of drugs, they may be systematically different from patients with no prior exposure. It may also introduce left censorship bias (see file 2 in online supplement). Investigators should describe prior exposure in their exposed cohort and discuss the implications of their selection.
Selection of the comparison cohort is critical to interpretation of the adverse event rate in the exposed cohort. The comparison cohort provides a reference with regard to the probability of the outcome in an unexposed but otherwise comparable population. While study design and data availability may not allow the ‘perfect’ comparison cohort, authors should explain their reasoning for selection of the comparison group and any limitations. When comparators are defined by other drug use, authors should, as with the exposed cohort, particularly consider the pattern of risk with time (see file 2 in online supplement).
In real-life biologics registers, patients start and stop drug treatments, switch drugs and restart previous treatments. Individual patients who were exposed can contribute person time to the unexposed cohort (and vice versa) only assuming time spent in one cohort carries over no risk to the other cohort after switching. However, the reason for switching biological therapy may have an implication upon the risk of adverse events (see file 2 in online supplement). Similarly, when patients restart the same therapy, ‘islands’ of treatment exposure time can only reasonably be grouped together if occurrences in the interval between exposures do not affect the risk of the outcome with subsequent treatment. Taking into account the full effects of switching between treatment exposures during follow-up may pose a real analytical challenge.
Variables
Treatments administered on a daily basis provide intuitive start and stop dates. For treatments administered at infrequent but set intervals, the stop date may be defined as the first missed dose or a set number of half lives beyond the last given dose. If the stop date is defined as the last given dose, events which immediately follow the last administration would not be attributed to the drug in the analysis, grossly underestimating the event rate.4 For treatments that do not have a set time period for the next treatment or differing pharmacological durations of action between individuals (eg, rituximab in RA), it may be necessary to define the stop date as the last given dose, but then to consider a lag window of risk (see below) for analysis.
Statistical methods
Outcome can be attributed to drug exposure from any time period ranging from ‘actively receiving drug’ to ‘ever exposed’. No ‘correct’ model exists for attributing outcomes to drug exposure. Indeed, it is likely that different analysis models are required for different outcomes under investigation. Our experience is that each investigating team has, to date, devised their own unique model for outcome attribution. Comparability between studies could be improved by harmonising the models by which events are attributed to drugs. The models ‘on drug’ and ‘ever exposed’ represent two ends of the spectrum. A third model connecting the other two is ‘on drug plus a lag window’, where the duration of the lag window can be adjusted. This lag risk window following drug cessation allows time for the drug to clear from the body, as well as events to present clinically. The duration can be varied for different drugs or outcomes—for example, it can be increased for a disease with a long latency such as malignancy. Even with such flexibility, this model would improve readers' ability to interpret differences between studies. We strongly advocate adoption of these models. Illustrations and examples are given in file 2 in the online supplement.
Authors should describe and justify which model of outcome attribution they have used. It may be reasonable to indicate the sensitivity of the results to these definitions. When investigators plan to explore an outcome previously published by a different group, they should consider using a similar analysis model and outcome definitions as was used in the index study. For each set of results presented from a model, it is important that the crude number of events and person time are also presented as these would vary between models. Sensitivity analyses using different models may provide additional information about the influence of exposure time on the relation between exposure and outcome.
Descriptive data
Recruitment to biologics registers usually occurs over years. With protracted inclusion of subjects, there will be a spectrum of cumulative exposure time. Patients can also be lost to follow-up at different time points during the course of a study. Results should include not just the number of patients included in the analysis, but the total treatment experience in the treated and comparison cohort, reasons for drug discontinuation (if available) as well as average treatment duration for both cohorts. Description of period of inclusion, duration of drug exposure and changes to the cohorts with time are particularly important for outcomes that have time-varying risk (see file 2 in online supplement).
Outcome data
An important consideration when presenting data is the time dependence of the outcome. For example, infusion reactions occur immediately whereas other outcomes such as malignancy may require a latency period until they become detectable. Analysis can be stratified by time bands. It is useful to graphically illustrate the time dependence of the outcome.
Main results
Understanding risk may be difficult for patients and clinicians.11 Some complex risk measures are meaningless in isolation—for example, relative risks cannot be interpreted without the context of baseline risk. Biologics registers should seek to present absolute measures (eg, events per 1000 patient-years) as well as relative measures (eg, relative risks). Supplementary ways of presenting risk should also be considered, such as ‘numbers needed to treat/harm’ (see file 2 in online supplement). The crude association between exposure and outcome should be presented as well as estimates following statistical adjustment. This can be important when clinicians correlate results with their clinical experience. Whereas crude association reflects the association perceived in clinical practice, the adjusted is typically better at reflecting a specific biological research hypothesis. For perspective as well as clinical utility, it is helpful to report the intrinsic association between each confounder and the outcome.
Along with overall relative risk measures, time-dependent measures of incidence and relative risks need to be provided. These should be presented in a coordinated way with changes to cohort numbers so that the reader can easily identify the numerator and denominator during each specified time period.
Other analyses
The association between exposure and outcome is seldom equal across all individuals under study. Certain characteristics (eg, presence of rheumatoid factor) might be important predictors or clinical determinants of risk (see file 2 in online supplement). Such effect measure modification can be investigated and quantified using a series of methods.
Similar to sensitivity analyses for different categories of the study population and for different definitions of exposure, researchers may consider sensitivity analyses for different definitions of outcome. Comprehension of how outcome definition or validation criteria influence the results provides useful information on the robustness of the main result (see file 2 in online supplement).
Limitations
Any true causal association can be obscured in an observational study by multiple potential biases.12 Authors should carefully examine whether systematic differences have occurred between the treated group and the comparison group with regard to the characteristics of patients (disease severity, comorbidity, drug use, etc), the assessment and verification of the outcome or loss to follow-up. Selection bias and confounding by disease severity are particular threats to the interpretation of biologics registers (see file 2 in online supplement). Careful consideration of bias and confounding, as well as possible residual or unmeasured confounding, is critical within the discussion.
Discussion
The establishment of a number of biologics registers in rheumatology has provided the opportunity to compare and contrast methodologies of establishing registers, ongoing data collection and analysis and reporting of results. When different groups have addressed the same scientific question and reported results, the varied publications provide a unique opportunity to scrutinise the conduct of such registers. We have the chance not only to name differences, but also to define areas where research communities might harmonise practice to advance future studies.
While based on the experience of biologics registers in rheumatology, the principles highlighted in this document should be applicable across different diseases. Of course, different geographical or disease settings will generate fundamental differences in the ability to establish registers and/or perform observational studies on drug treatment. Thus, the two lists assembled should be viewed as guidance rather than a set of universal rules of conduct. Imperative or not, it is important that consideration of these items takes place in the planning phase of a register as well as at the time of starting analysis. The points to consider in this paper should benefit not just scientists working directly with registers or register data, but also reviewers and readers of their reports.
In a time of increasing treatment intensity, elevated treatment goals and an increasing number of novel therapies in rheumatology, the complexity of assessing individual drug safety profiles will rise. Sequential or overlapping treatment will further necessitate transparent and carefully conducted observational drug studies. It is our hope that the points to consider presented here will help us meet these demands.
References
Supplementary materials
Web Only Data ard.2009.125526
Files in this Data Supplement:
Footnotes
-
Competing interests None. Johannes WJ Bijlsma was the handling editor for this manuscript.
-
Provenance and peer review Not commissioned; externally peer reviewed.