Article Text
PDF
Abstract
Objectives To assess variation in and determinants of rheumatologist guideline adherence in patients with rheumatoid arthritis (RA), in daily practice.
Methods In this retrospective observational study, guideline adherence in the first year of treatment was assessed for 7 predefined parameters on diagnostics, treatment and follow-up in all adult patients with RA with a first outpatient clinic visit at the study centre, from September 2009 to March 2011. Variation in guideline adherence was assessed on parameter and rheumatologist level. Determinants for guideline adherence were assessed in patients (demographic characteristics, rheumatoid factor (RF) and/or anti-cyclic citrullinated peptide antibody (aCCP) positivity, erythrocyte sedimentation rate, erosive disease, comorbidity and the number of available disease modifying anti-rheumatic drug (DMARD) treatment options) and rheumatologists (demographic and practice characteristics, guideline knowledge and agreement, outcome expectancy, cognitive bias, thinking style, numeracy and personality).
Results A total of 994 visits in 137 patients with RA were reviewed. Variation in guideline adherence among parameters was present (adherence between 21% and 72%), with referral to the physician assistant as lowest scoring and referral to a specialised nurse as highest scoring parameter. Variation in guideline adherence among rheumatologists was also present (adherence between 22% and 100%). Patient sex, the number of DMARD options, presence of erosions, comorbidity, RF/aCCP positivity, type of patient and the rheumatologists' scientific education status were associated with adherence to 1 or more guideline parameters.
Conclusions Guideline adherence varied considerably among the guideline parameters and rheumatologists, showing that there is room for improvement. Guideline adherence in our sample was related to several patient and rheumatologist determinants.
- Rheumatoid Arthritis
- Quality Indicators
- Treatment
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key messages
What is already known about this subject?
Guideline adherence to tight control treatment strategies is often suboptimal when assessed in predefined rheumatoid arthritis (RA) cohorts.
What does this study add?
This is one of the first studies to assess RA guideline adherence by rheumatologists in daily practice, and to explore associations between guideline adherence, and patient-related and rheumatologist-related determinants.
Previous studies on RA guideline adherence focused on therapy change and/or disease modifying anti-rheumatic drug prescriptions, whereas we now included a broader range of parameters covering diagnostics, treatment and follow-up.
In this study, adherence percentages varied considerably among the 7 parameters (range: 21–72%) and 14 rheumatologists (range: 22–100%), with several patient-related and rheumatologist-related determinants explaining part of the inter-rheumatologist variation.
How might this impact on clinical practice?
The results of this study can serve as a stepping stone for further research into effective interventions to improve rheumatologists' guideline adherence, allowing patients to benefit from optimal RA care.
Introduction
Many guidelines and recommendations on optimal care for patients with rheumatoid arthritis (RA) have been developed to help clinicians choose the best diagnostic and therapeutic strategies for their patients with RA. All major RA treatment guidelines are now based on tight control principles, where monitoring of disease activity and changing treatment if a preset target is not reached are essential.1–3 Adherence to these tight control principles, preferably combined with the use of a specific treatment guideline, results in lower disease activity and less functional damage compared with usual care.4–6
In view of the evidence supporting the benefit of adhering to tight protocolised control strategies, it is disappointing that current guideline adherence is still suboptimal, as observed in multiple studies on this topic.7–13 Unfortunately, these studies focus on disease modifying anti-rheumatic drug (DMARD)-related treatment recommendations only, disregarding the fact that other aspects of RA care are also important. Furthermore, these studies are not performed in daily practice, but in predefined cohorts, using subsets of patients with RA. Therefore, the first aim of this study is to gain more insight into guideline adherence of rheumatologists in daily practice, using a broader set of guideline adherence parameters than before.
Our second study aim is to gain insight into determinants of guideline adherence. In order to improve guideline adherence, it is first necessary to understand the determinants that influence adherence. Knowledge on these determinants could then be used to develop targeted interventions, as evidence suggests that this leads to better intervention effects.14 Although knowledge on determinants of guideline adherence is not yet available from studies within rheumatology, studies outside rheumatology suggest the importance of various determinants, explaining the observed variation in guideline adherence among both, hospitals and physicians.15 Examples of such determinants are knowledge and cognitions of individual healthcare professionals and patient factors.15
All in all, data on RA guideline adherence in daily routine and its association with potential determinants are still lacking. Therefore, we aimed to (1) assess RA guideline adherence in daily clinical practice, (2) assess variation in guideline adherence on parameter and rheumatologist level and (3) explore the impact of rheumatologist-related and patient-related determinants on guideline adherence.
Material and methods
Study design
An explorative, retrospective observational multilevel cohort study was performed. Guideline adherence is behaviour executed by a rheumatologist, but it is measured in patients who visit the hospital. Hence this study has three different levels: outpatient clinic visits (level 1) are nested within patients (level 2) who are in turn nested within rheumatologists (level 3). This is also reflected in the data collection and measurement: guideline adherence is measured on patient or visit level (data collection on visit level), whereas the possible determinants of guideline adherence were measured either on rheumatologist or patient level.
Setting
This study was conducted at the Rheumatology Department of the Sint Maartenskliniek, a large clinic in the Netherlands, specialised in rheumatology, rehabilitation medicine and orthopaedics. In this centre, a local, tight control-based, RA treatment guideline was initially put into use in 2007. At the same time, supportive actions were undertaken to aid rheumatologists in following the new guideline. First, specialised nurses were available to provide patient education, discuss disease coping and to assess disease activity before the visit with the rheumatologist (nurse-led assessment of the Disease Activity Score in 28 joints (DAS28)). Second, after a referral to the physician assistant (PA) by the rheumatologist, patients were seen in alternating fashion by the PA and rheumatologist in order to share care between the two. The PAs can independently make treatment decisions but they work under the supervision of a rheumatologist and, at the time of this study, were not allowed to prescribe medication.
Participants
All 14 rheumatologists working at the study centre from September 2009 through July 2012 were eligible for participation. Rheumatologists who did not work the full period were excluded; no other exclusion criteria were set. Consent from all participants was sought while explaining the study during a regular staff meeting.
We included all patients 18 years of age and older, diagnosed with RA (International Classification of Diseases, Ninth Revision (ICD-9) code 714.0), treated by one of the included rheumatologists and having had a first outpatient clinic visit at the study centre anytime from September 2009 to March 2011. Patients with either new or established RA were included, as long as their first visit to the study centre took place during the given time period. If patients were seen as second opinions they were only included if treatment was fully taken over by the study centre. After inclusion, all visits in the first year of treatment at the study clinic were used to assess guideline adherence (figure 1). This means that the follow-up period lasted until March 2012.
Study timeframe. *This included start of data collection and informing rheumatologists at the study centre about this study.
Guideline adherence measures and data collection
As guideline adherence is multidimensional and cannot be expressed by a single outcome measure, we defined a set of seven different parameters to measure various quality aspects of RA care. These parameters are based on the quality indicators stated in the Dutch national RA treatment guideline.16 As the local RA guideline used in the study centre is an adapted version of the Dutch national guideline, the selected parameters were adapted accordingly. This resulted in a set of seven guideline adherence parameters concerning three main themes (diagnostics, treatment, follow-up and shared care). All parameters are reported as dichotomous outcomes (‘yes’ or ‘no’) but, depending on the type of parameter, this is carried out either at visit or at patient level. All guideline adherence parameters are described in table 1. Online supplement 1 provides a more extensive version of this table, including corresponding treatment recommendations.
Supplemental material
Guideline adherence parameters
As mentioned before, all parameters were measured during the first year of treatment at the study centre. So, after a patient was included (from September 2009 to March 2011) all visits in the next year were used to measure guideline adherence.
To calculate the different parameters, the following data from every visit in the first year of treatment were collected: date and type of visit, name of treating rheumatologist, presence of a nurse-led DAS28 assessment, DAS28 score (using erythrocyte sedimentation rate (ESR)), functional status by Health Assessment Questionnaire (HAQ), clinical judgement of disease activity, radiographs ordered, current medication use (conventional and biological DMARD, glucocorticoids and/or non-steroidal anti-inflammatory drugs (NSAIDs)), referral to a specialised nurse and referral to a PA. Using predefined algorithms, the seven guideline parameters were calculated using the aforementioned data.
Determinants of guideline adherence and data collection
Determinants of guideline adherence were assessed on two different levels: patient and rheumatologist. On the patient level, eight determinants were collected at baseline: age, gender, type of patient (new or second opinion), rheumatoid factor (RF) and/or anti-cyclic citrullinated peptide antibody (aCCP) positivity, ESR, presence of erosive disease, relevant comorbidity and the number of available DMARD treatment options. The last determinant provides the number of conventional and biological DMARDs that patients have not yet used, but could be prescribed in the future in case of treatment failure on the current DMARD.
On the rheumatologist level, information on five demographic and practice determinants was collected (age, gender, PhD, years of work experience as a rheumatologist and percentage of direct patient contact per week at the outpatient clinic). Furthermore, all participating rheumatologists were asked to complete self-developed questionnaires on guideline knowledge and agreement, and outcome expectancy. In addition, existing and validated questionnaires on cognitive bias, thinking styles, numeracy and personality traits were administered.17–20 Some of the included questionnaires expressed their score on ≥1 subscale, resulting in 14 determinants being calculated from seven questionnaires (table 2 and online supplement 2). All questionnaires, of which the invitation to participate was sent to the rheumatologists in July 2012, were web-based. After 2 weeks, reminders were sent to all rheumatologists who had not yet completed the questionnaires.
Supplemental material
Questionnaires used to measure rheumatologist-level determinants
Data sources
All data needed to calculate the guideline adherence parameters were retrospectively retrieved from paper hospital charts, using paper case report forms and a patient list generated from the administrative hospital database. During this chart review, the patient-related determinants were also collected. After chart review, all data were entered into an electronic database and anonymised. All rheumatologist-related determinants were collected using the questionnaires mentioned in the previous section and scores were also entered into an electronic database. For the purpose of the study, anonymising the rheumatologist data was not possible.
Statistical analysis
Results on primary outcome measures (guideline adherence parameters) are reported as percentages with the accompanying absolute numbers. For the questionnaire scores and remaining variables, means and SD or medians and IQRs are provided.
Owing to the hierarchical structure of our study (repeated measures on the same patient and patient nested within rheumatologists) we performed linear or logistic multilevel regression analysis when analysing the relation between the guideline adherence parameters and determinants. Depending on the type of parameter (outcomes on patient or visit level), two or three levels were included in the analyses. For the parameters—radiographs ordered, preferential DMARD order, referral to a specialised nurse and referral to a PA (patient level)—each parameter had the same score for every visit within one patient. For example, patients should be referred to a PA within the first year of treatment, meaning that this parameter is scored only once per patient, taking into account all visits during the study period. Multilevel analysis for these parameters only accounted for clustering within rheumatologists (two-level model). For the other parameters (therapy change, nurse-led DAS28 assessments and correct visit intervals; visit level) multilevel analysis also accounted for clustering within patients. This extra level was added because the parameter score per visit could differ within patients. For example, nurse-led DAS28 assessments were either conducted or not conducted during the various visits.
Multilevel analysis started with adding all patient determinants to the model. Then, one by one, the least significant determinant was deleted from the model until all remaining determinants were significant (p<0.05). Next, the rheumatologist determinant with the highest correlation was added to the multilevel regression model and, if significant, included in the final regression model. This process was repeated with the rheumatologist determinant with the second highest correlation. Depending on the p value of this determinant in the model, the analysis was either stopped (final model) or another determinant was added. This method was chosen because the number of rheumatologists was relatively small compared with the number of rheumatologist-related determinants.
Only parameters and determinants with enough variation among rheumatologists were analysed for associations between them. In case of floor or ceiling effects, a determinant was omitted from further analysis. Results are presented as ORs with the corresponding 95% CI, p value and explained variation (%). Explained variation was calculated using the method described by Snijders and Bosker.21
Statistical analyses were performed using STATA V.13.0, except the multilevel analysis, this was carried out using SAS V.9.2.
Ethical approval
This study was approved by the local research committee at the study centre (RR-105-PP). Although no written informed consent was obtained from the rheumatologists, they were informed beforehand about this study and asked if they would participate. It was made clear to them that they could withdraw from the study at any time, without providing a reason.
As this was a quality assessment performed in the hospital where the first two authors of this study worked, no written informed consent was needed from the patients. In addition, data collection was made by the first author and, directly after chart review, all patient data were anonymised.
Results
Rheumatologist and patient characteristics
All 14 eligible rheumatologists (46.2% female; mean age 47.6±10.0 years) participated in this study. All questionnaires were returned by all rheumatologists, except for the questionnaires on guideline knowledge, and agreement and outcome expectancy, which one rheumatologist did not complete.
According to the hospital database, 241 patients with an ICD-code of RA were seen for the first time at the study clinic, from September 2009 to March 2011. Sixty-one patients were excluded because they turned out to be second opinion patients of whom treatment was not taken over by the study centre. Additionally, 43 patients were excluded because either charts were missing (n=9), patients were not seen by an included rheumatologist (n=11), chart review revealed a diagnosis other than RA (n=16) or patients had deceased (n=7). The remaining 137 patients with RA (67.2% female; mean age 58.9±14.1 year), with a total of 994 visits, were included in this study. Roughly half the patients had not been seen by a rheumatologist before (46%); the remaining patients had been treated before by a rheumatologist outside the study centre, and visited the study centre for a second opinion. This led to a combination of new and established RA, as reflected in the median disease duration (0; IQR 0–7 years). In table 3, the baseline characteristics of both, rheumatologists and patients, are stated.
Baseline characteristics of included rheumatologists and patients
Guideline adherence parameters
Adherence to the different guideline adherence parameters varied between 21% and 72% (table 4). The best scoring indicator was ‘referral to a specialised nurse’, with 72% of the patients being referred to such a nurse. Ordering of radiographs and changing therapy in case of active disease was performed in approximately two-thirds of patients or visits, respectively. The remaining parameters had adherence percentages between 20% and 40% (PA referral, DMARD prescription, intervals between visits and nurse-led DAS28 assessment).
Guideline adherence percentages
As described in table 4, not all guideline parameters apply to all patients or visits. For example, this applies to the parameter ‘therapy change in case of active RA’. A DAS28 and/or clinical judgment was available in 622 visits (63%) and, in 285 of these visits, active disease (DAS28>3.2 or a corresponding judgement from the treating rheumatologist) was present (46%). In 191 of those visits (67%), the rheumatologists decided to change the patient's medication (parameter therapy change in case of active disease).
In addition to the aforementioned variation among guideline adherence parameters, variation was also observed among rheumatologists. The largest difference among rheumatologists was seen in the parameter concerning radiograph ordering, with adherence percentage of individual rheumatologists varying from 22% to 100%. The least variation was seen in the parameter on correct intervals between visits, with adherence percentages varying from 11% to 43%.
Determinants of guideline adherence
All guideline adherence parameters showed enough variation and no floor/ceiling effects were present, therefore all parameters were included in the multilevel analyses. However, five determinants (general guideline knowledge, general guideline agreement, specific guideline agreement, outcome expectancy and numeracy) were not included in the analyses due to lack of variation in the scores and/or ceiling effects (table 3).
For the remaining determinants, eight associations with five different parameters were found (table 5). The preferential order of DMARD prescriptions was adhered less to in case of more available treatment options. Furthermore, referral to a specialised nurse was less likely if patients had erosive disease and comorbidity at baseline. Females, aCCP-positive and/or RF-positive patients and second opinions had less visits combined with a nurse-led DAS28 assessment. Correct intervals between visits were also less likely if a patient was seen for a second opinion.
Multivariate associations between guideline adherence parameters and P-related and R-related determinants
Only one parameter was associated with a rheumatologist-related determinant: rheumatologists with a PhD degree or pursuing a PhD were more likely to refer their patients to a PA. Personality, thinking styles and cognitive bias did not impact rheumatologists' adherence to any of the guideline adherence parameters.
The explained variance of the models was low to moderate. The lowest explained variance (2.5%) was seen in the model on correct intervals between visits, and the highest (12.0%) in the model on PA referral.
Discussion
Our results show that guideline adherence percentages varied considerably among parameters, suggesting suboptimal guideline adherence on at least some guideline recommendations. Furthermore, adherence also varied among rheumatologists, and several rheumatologist and patient-related determinants (rheumatologists' scientific education status, patient sex, number of DMARD options, presence of erosions, comorbidity, RF/aCCP positivity, type of patient) were found to be related to rheumatologists' guideline adherence.
Besides being one of the first studies in rheumatology assessing guideline adherence in daily practice, other strengths of this study are the inclusion of a wide range of guideline adherence parameters, and the multilevel association analyses between these parameters and determinants. However, our study has some limitations. First, being a retrospective study with chart review as the main data source, it is possible that information has been missed due to the fact that not everything was well documented in the charts. However, the advantage of our retrospective design is that guideline adherence could not have been influenced by the study itself. Second, the sample size was relatively small, with only 14 participating rheumatologists. Nonetheless, if we have missed associations due to a lack of power, these associations are probably not very strong. Third, this study was conducted in only one centre in the Netherlands, probably limiting generalisability. Nevertheless, our observation that guideline adherence is suboptimal is most likely to be generalisable, as earlier studies made the same conclusions. Only, our estimates on the degree of guideline adherence might be less generalisable. Furthermore, due to the single centre design, we were not able to assess the influence of organisational factors on guideline adherence. As the study centre already implemented some supportive actions to increase adherence, results in a centre without these actions might be different. Last, the single-centre design and the homogeneous population within this centre contributed to the exclusion of some of our determinants due to ceiling effects or lack of variation.
In our study, guideline adherence varied from 21% to 72% and, as no absolute norms on optimal guideline adherence exist, we can only use relative norms to judge if guideline adherence in this study was optimal. First, the adherence percentages of our best scoring parameters can be used as a relative norm. So, the observed level of adherence to the three highest scoring indicators (radiograph ordering, specialised nurse referral, therapy change; adherence 66–72%) was probably optimal. Furthermore, aiming for 100% adherence is not feasible due to, for example, patient comorbidity or medication side effects.
Second, we can compare our results with other studies. However, since previous studies have primarily focused on therapy recommendations (DMARD prescription and therapy change in case of active disease), this makes comparison with existing data impossible for all our parameters. With regard to DMARD prescriptions, the 23% guideline adherence we found seems to be on the lower end of the spectrum. Another study on this subject observed adherence percentages to the American College of Rheumatology (ACR) DMARD treatment guidelines of 24% to 90%, depending on the type of DMARD used, disease activity and prognosis.11 However, the lower adherence percentages in our study could probably be explained by the more strict definition we used. For example, the ACR guideline names methotrexate as the first choice DMARD, with combination therapy depending on disease activity, prognosis and disease duration. In contrast, according to our local guideline, all new patients should be started on methotrexate and hydroxychloroquine combination therapy.
With regard to therapy change in case of active disease, we can compare our results with two previous studies. A study by Fransen et al 9 on patients with established RA used the same DAS28 threshold (3.2) at which therapy should be changed, as our study did, observing an adherence percentage of 20%. Although Fransen et al looked only at DMARD therapy change whereas we included corticosteroid use as well, the adherence of 67% that we found is substantially higher. The result of the second study, by Vermeer et al,13 with an adherence of 58%, is more in line with our results, although it only included DMARD therapy change and was limited to early RA. Concerning therapy change, it should be mentioned that our centre has participated in a guideline adherence study before. In that particular study, therapy was changed in 33% of the visits with active disease, compared to 67% now.22 This large improvement in guideline adherence is most likely caused by the introduction of the local RA guideline and the supportive actions thereafter.
Besides the assessment of guideline adherence, we also examined whether patient-related and rheumatologist-related determinants were associated with guideline adherence. On these associations some interesting observations can be made. For example, in qualitative studies, factors such as erosive disease, comorbidity and RF/aCCP status, are often mentioned by rheumatologists as important reasons to either intensify or not intensify treatment.23 Therefore, we expected to find associations between these determinants and the guideline parameter on therapy change in case of active disease. Although we did not observe this association, we observed associations between number of treatment options, erosive disease, comorbidity and RF/aCCP status, and the parameters on DMARD prescription, referral to a specialised nurse and nurse-led DAS28 assessments. This implies that patient factors could, justly or unjustly, influence more decisions than treatment intensification only.
With regard to the rheumatologist-related determinants, it is notable that only one association between a rheumatologist determinant (PhD) and a guideline adherence parameter (PA referral) was found. This was especially surprising as factors such as knowledge are frequently mentioned as a potential determinant of guideline adherence.15 ,24 This might imply that rheumatologist-related determinants did not play a large role in our sample, but further studies on this subject are needed as guideline adherence is probably determined by a complex interplay of facilitators and barriers that makes it hard to capture.
Owing to the explorative design of our study, replication of our results is warranted in other settings both inside and outside the Netherlands. However, the suggestion from our results that rheumatologists do not always practise what they preach can be used more widely. It seems that, despite the current focus on treat to target principles in RA literature, these principles are not automatically applied in daily practice.
This study provides an example for other centres to measure their quality of care and the determinants found in our sample might be reckoned with in future interventions. Recent developments around nationwide registries, such as the Rheumatology Informatics System for Effectiveness (RISE) registry, can facilitate measurements by providing real-time feedback on important aspects of quality of care.25 Information gained from quality of care studies or registries can then serve as benchmark information for hospitals and individual physicians.26 Furthermore, we would advocate for more attention of researchers and policymakers towards implementation of RA guidelines and quality of care. Besides replicating our results in larger studies, two important topics for future research are the identification of determinants influencing adherence and exploration of the impact of separate adherence parameters on patient outcomes (eg, Does referral to a PA lead to more patients being in remission?). Preferably, such studies would be conducted in a large, nationwide multicentre setting in order to aid precision and generalisation of study results. These steps are crucial to gain insight into the most effective and feasible interventions to help rheumatologists better adhere to RA management guidelines and to improve patient outcomes in daily practice. Only then can patients benefit from the large body of evidence that already exists.
Acknowledgments
The authors would like to thank all the rheumatologists who participated in this study for their willingness to complete the questionnaires. Furthermore, they would like to thank Reinier Akkermans for his help with the multilevel analyses.
References
Footnotes
Contributors NL, AAdB and RFvV contributed to the study design. NL collected the data. NL, AAdB and MEJLH participated in the data analysis. NL, AAdB, MEJLH and RFvV assisted in the interpretation of the data. The manuscript was written by NL under the supervision of AAdB, MEJLH and RFvV. All the authors revised the draft version of the manuscript, and read and approved the final version of the manuscript. NL is the guarantor.
Competing interests RFvV has received grants and personal fees from AbbVie, BMS, GSK, Pfizer, Roche, UCB, Biotest, Crescendo, Janssen, Lilly, Merck and Vertex, outside the submitted work.
Ethics approval Ethics committee of the Sint Maartenskliniek.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Patient-level data, full data set, technical appendix and statistical code are available on request from the corresponding author. Consent from the patients was not obtained but the presented data have been anonymised and risk of identification is low.