Figures
Abstract
Objectives
To validate and optimize a referral rule to identify primary care patients with chronic low back pain (CLBP) suspected for axial spondyloarthritis (axSpA).
Design
Cross-sectional study with data from 19 Dutch primary care practices for development and 38 for validation.
Participants
Primary care patients aged 18-45 years with CLBP existing more than three months and onset of back pain started before the age of 45 years.
Methods
The referral rule (CaFaSpA referral rule) was developed using 364 CLBP patients from 19 primary care practices and contains four easy to use variables; inflammatory back pain, good response to nonsteriodal anti-inflammatory drugs, family history of spondyloarthritis and a back pain duration longer than five years. This referral rule is positive when at least two variables are present. Validation of the CaFaSpA rule was accomplished in 579 primary care CLBP patients from 38 practices from other areas. Performance of the referral rule was assessed by c-statistic and calibration plot. To fit the final referral rule the development and validation datasets were pooled leading to a total study population of 943 primary care participants.
Results
The referral rule was validated in 579 patients (41% male, mean age 36 (sd7.0). The percentage of identified axSpA patients was 16% (n=95). External validation resulted in satisfactory calibration and reasonable discriminative ability (c-statistics 0.70 [95% CI, 0.64-0.75]). In the pooled dataset sensitivity and specificity of the referral rule were 75% and 58%.
Citation: van Hoeven L, Vergouwe Y, de Buck PDM, Luime JJ, Hazes JMW, Weel AEAM (2015) External Validation of a Referral Rule for Axial Spondyloarthritis in Primary Care Patients with Chronic Low Back Pain. PLoS ONE 10(7): e0131963. https://doi.org/10.1371/journal.pone.0131963
Editor: James T. Rosenbaum, Oregon Health & Science University, UNITED STATES
Received: February 6, 2015; Accepted: June 8, 2015; Published: July 22, 2015
Copyright: © 2015 van Hoeven et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: An unrestricted research grant was provided by AbbVie Netherlands, grantnumber IMM-10-0147 (www.abbvie.nl). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: AbbVie funded this study. This does not alter the authors’ adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
Axial spondyloarthritis (axSpA) is relative new term in the field of rheumatology. It is a chronic inflammatory joint disease, that is potentially disabling and characterized by chronic low back pain (CLBP). [1] AxSpA is associated with increased morbidity, mortality, high health care costs and reduced work productivity. [2, 3] Quality of life and work participation can be improvement with effective treatment; non-steroidal anti-inflammatory drugs (NSAIDs) and biologicals.[4] This treatment is even more effective when it is given early in the disease course [5]. Nevertheless there is a delay of 4–9 years between the first CLBP symptoms and the final diagnosis of axSpA. [6, 7] This delay can be explained by the difficulty for primary care physicians to recognize an axSpA patient in the large amount of CLBP patients seen in primary care.
Low back pain (LBP) is one of the most common health problems and it is worldwide the largest contributor to the overall amount of years lived with disability (YLDs) causing a large burden for patients, health systems and society. [8, 9] Around 10% of LBP complaints persists for more than 12 weeks and become chronic. [10] In most countries CLBP patients are first seen by their primary care physicians. Guidelines with red and yellow flags are used to diagnose, treat and if necessary refer CLBP patients. [11]. These guidelines do not include a flag or referral recommendation specific for axSpA. The lack of a specifically axSpA flag is notable since a number of recent studies showed that up to 40% of the CLBP complaints, if patient are referred by pre-defined criteria, can be explained by axSpA. [7, 12–17] In addition to studying prevalence these studies also proposed different referral strategies. Referral strategies for axSpA aim to achieve earlier referral of patients suspected for axSpA by primary care physicians. However most of the published referral rules were not easy to use, costly, or developed in secondary care patients. This pre-selection of patients makes it hard to implement these referral strategies in primary care practice. Furthermore most published referral strategies are merely based on development studies so no external validation took place, an important step for deriving a clinical useful referral strategy. [18] In 2014 we published the CaFaSpA referral rule, a referral strategy for axSpA developed in primary care patients with CLBP and applicable for primary care physicians. [7] In this study we want to externally validate and optimize the performance of this CaFaSpA referral rule in another, independent population of young primary care CLBP patients.
Material and Methods
Study design and data source
We did a cross-sectional study in a large population of primary care CLBP patients from June 2011 to June 2012, the acronym of the study was the CaFaSpA (Case Finding Axial SpondyloArthritis) study. Primary-care group practices in the Rotterdam and The Hague area in the Netherlands were informed about the study and invited to participate. In total 38 GPs participated, who represented a source population of about 28.842 patients, ages 18–45 years. Potential participants with LBP were selected from the GP databases using the International Classification of Primary Care (ICPC) code L03, standing for low back pain symptom/complaint excluding radiation. [19].
From the 28.842 primary care records, 2597 (9%) patients ages 18–45 years were identified who had ever been registered by the ICPC code L03. Those 2597 patients were invited to participate by a letter on behalf of their GP. Responding participants were checked for eligibility during a telephonic interview by a research assistant. Inclusion criteria were current low back pain existing for more than 12 weeks, good understanding of the Dutch language and no contraindications for MRI (i.e. pregnancy, claustrophobia, pacemaker). Patients were excluded if there was a explainable cause for the back pain, such as a hernia nuclei pulposi or a trauma.
Ethics statement
Written informed consent was obtained from all participants at the research center before any assessment was performed. Ethics approval from the St. Elisabeth Hospital in Tilburg, the Netherlands was obtained (NL3571806011).
Clinical evaluation
All participants were asked to complete the ASAS [1] questionnaire on inflammatory back pain (IBP), before any clinical and/or radiological evaluation was done. This questionnaire comprised of five questions related to back pain. A positive ASAS questionnaire was achieved when four out of five questions were answered positively. The outcome the ASAS questionnaires was reported in a binary value; positive or negative. Furthermore participants completed the BASDAI[20] and ASDAS [21] questionnaire, both measure the disease activity of axSpA, a higher score indicates a higher disease activity. Also the Roland Morris disability questionnaire (RMDQ) was completed. [22] The RMDQ is a measure of disability caused by the LBP. Higher numbers on a 24-point scale reflect greater levels of disability.
Within a rheumatology setting an experienced research nurse obtained a clinical history including axSpA features, namely IBP, arthritis, psoriasis, enthesitis, dactylitis, uveitis, Crohn’s disease/colitis, good reaction to non-steroidal anti-inflammatory drugs (NSAIDs) and a positive family history of SpA.
The ‘red flags’ used by primary care physicians, standing for typical signs or symptoms that are frequently associated with specific LBP were also checked.[23] A description of the red flags is available in S1 Table. All assessments followed the definitions described in the ASAS handbook.[1] Statistical comparisons between clinical features of axSpA patients and CLBP patients were made by the Student t test or X2 test, when appropriate.
Blood was drawn from all patients, irrespective of the research nurse’s opinion of clinical diagnosis of axSpA or IBP, for the Erythrocyte Sedimentation Rate (normal range 0–15 mm Hg/min), C-reactive protein (normal range 1–10 mg/l) and HLA-B27 typing.
Image evaluation
All patients underwent image evaluation by X-ray and MRI, again irrespective of the research nurse’s opinion of clinical diagnosis of axSpA or IBP. Sacroiliac joints (SIJ) were scored according to the modified New York criteria (from 0 normal, to 4 complete fusion), using conventional pelvic radiographs in the anterior-posterior view.[24] A score of 0, 1 or 2 unilateral was considered normal, while bilateral grade 2 or unilateral grade 3 or 4 was classified as positive. A definitive diagnosis of sacroiliitis on MRI was made according to the ASAS criteria: presence of a minimum amount of bone marrow edema (one lesion in at least two adjacent slices or more than one lesion in at least one slice).[25] Images were read by one out of two trained radiologists, blinded for patient identity, clinical and laboratory data. If one of the radiologists doubted the score, the two observers discussed the scan and came to consensus.
Clinical outcome definition
Patients were classified as axSpA according to the ASAS criteria for axial spondyloarthritis. [25] Definite axSpA can be accomplished by the imaging arm; sacroiliitis on imaging (MRI or X-ray) plus ≥1 SpA feature, or by the clinical arm; no sacroiliitis on imaging but a positive HLA-B27 plus ≥2 SpA features. The SpA features are ASAS IBP, arthritis, (heel) enthesitis, uveitis, dactylitis, psoriasis, Crohn’s disease/colitis, good response to NSAIDs, family history for SpA, HLA-B27 positive and elevated C-reactive protein. A distinction between Ankylosing Spondylitis (AS) and non-radiographic axSpA (nr-axSpA) was made. The difference between AS and nr-axSpA is the presence of sacroiliits on plain radiographic of the sacroiliac joints (SI-joints). [1] AS comes with abnormalities on the X-ray consistent with sacroiliits, while nr-axSpA patients do not fulfill the imaging part of the modified NY criteria for AS.
Validated predictors
The CaFaSpA referral rule was previously developed with logistic regression analysis and internally validated with bootstrapping and corrected for over fitting by a shrinkage factor. [7] The original regression coefficients and odds ratios are given in S1 Table. The rule contained four dichotomous variables, the ASAS IBP questionnaire (positive vs negative), family history for SpA (positive vs negative), good response to NSAIDs (positive vs negative), LBP duration (≤ 5years vs >5 years). The ASAS IBP questionnaire is positive if at least four out of five questions are answered with yes, a positive family history means a first or second degree family member with axSpA, psoriasis, Crohn’s disease/colitis or uveitis. A good response to NSAIDs implies a clear improvement or disappearance of the low back pain, within 48 hours after the start of NSAIDs treatment.
External validation
For external validation of the referral rule, performance was assessed using discrimination and calibration measures.[26, 27] The ability to discriminate axSpA patients from CLBP patients was quantified by the c statistic, a measure for concordance. In binary outcomes, as in our model, the c statistic is identical to the area under the receiver operating characteristics (ROC) curve. Reasonable values for the area under the ROC curve range from 0.5 (no discrimination) to 1.0 (perfect discrimination). [28] Calibration is the agreement between the predicted probabilities and the observed frequencies and was assessed by estimating the calibration slope and intercept. The calibration slope is ideally 1 and reflects whether the effects of the predictors are on average correct. The calibration intercept indicates whether predictions are in general correct and is ideally 0. This intercept is assessed by fitting a logistic regression model with the linear predictor as an offset variable (setting the regression coefficients to 1). The analyses were performed using Stata version 13.0 software (Stata Corporation TX, USA) and R (version 2.15.2; The R Foundation for Statistical Computing).
Model updating
For the model updating, we decided to combine this validation dataset and the development dataset (CaFaSpA 1 study). [7] In 2014 the development study has been published which consisted of 364 CLBP patients from 19 primary care practices who had been included from January to July 2010 from the greater Rotterdam area in the Netherlands. By combining the datasets the model is based on more patients leading to more stable predictor effects.[29] First a logistic regression analysis was performed in the combined dataset. Subsequently we tested if adding new variables to the model leaded to significant improvement of the model Chi-square, a measure for overall performance of the model.
To present the model as a referral rule, a simple scoring system was made. We rounded the regression coefficients from the logistic regression analysis of the combined model. We estimated the sensitivity and specificity for several cut points. The positive predictive value (PPV) of the chosen cut point was calculated.
Results
Out of the 2597 invited patients with low back pain, 1161 patients (44.7%) responded (Fig 1). Of these 1161 responders, 480 expressed no interest in participating and 102 did not fulfilled the inclusion criteria. Informed consent was obtained from 579 participants.
Missing values
In the following variables missing values occurred: ASAS IBP questionnaire (1.5%) and laboratory parameters (0.5%). We assumed missing data occurred at random and performed single imputation of the variables used for the external validation.[30]
Characteristics of the study population
Table 1 shows the characteristics of the total study population, subdivided in axSpA and CLBP patients. Overall more women (59%) participated, the mean age was 35.9 years (sd 7.0) and the median duration of low back pain was 7 years (interquartile range 3–15 years). The overall prevalence of HLA-B27 was 6.2% (n = 36). The median VAS pain was 5 (IQR 3–7), the median BASDAI and ASDAS were respectively, 4.2 (IQR 2.3–5.9) and 2.3 (IQR 1.6–3.0). The median RMDQ score was 7 (IQR 3–13). The results of the red flags are available in S1 Table. No substantial differences in characteristics between the two studies were observed.
Percentage of identified axial spondyloarthritis patients
The percentage identified axSpA patients was 16.4% (n = 95), 95% CI: 13.5%-19.7% (Table 1). Within the axSpA cases 24 out of 95 (25%) were classified as AS and 71 (75%) as nr-axSpA. Twelve out of the 71 nr-axSpA patients (16%) fulfilled the ASAS criteria by the clinical arm, with a positive HLA-B27 status and at least two other SpA features.
Referral rule and combining datasets
Table 2 shows the discriminative ability of the original model (c-statistic 0.70, (95%CI 0.64–0.75)). The calibration slope was 0.77 indicating that the predictor effects were on average too large. The intercept of -0.49 indicates that predictions were on average too high, which is related to the lower percentage of identified axSpA cases in the current study (16.4%), compared to CaFaSpA 1 (23.6%). The c-statistic of the combined model is 0.70, with a smaller confidence interval (95% CI 0.66–0.74).
The predictor effects of the ASAS IBP questionnaire, family history and reaction to NSAIDs were similar or smaller in the validation data compared with CaFaSpA 1 (Table 3). The effect of LBP duration was not profound anymore as was also shown by the similar prevalence of axSpA in two different LBP duration groups (16.8% in LBP ≤5 years versus 16.2% in LBP >5 years).
We studied the additive effect of age and a dichotomized variable with easy to determine SpA features (arthritis, dactylitis, psoriasis, enthesitis, uveitis and inflammatory bowel disease, 0 = no SpA features present and 1 = ≥1 SpA feature present) in the combined data. Neither variable increased the model Chi-square significantly.
To provide a user friendly format of the prediction model, predictors with similar regression coefficients were given equal points in a simple scoring system. Fig 2 shows the combined model in this simple scoring system that can be used as a referral rule. A score of 0.5 was given to a symptom duration longer than 5 years. A positive ASAS questionnaire, a positive family history for SpA and a good reaction to NSAIDs all received one point. The cut point of 1.5 point was associated with a sensitivity of 75% and a specificity of 58% (Table 4). The yield of the referral rule expressed in the PPV is 30.2%. This means that 30.2% of the CLBP patients with a positive referral rule can be identified as axSpA.
Discussion
Our validation study confirms the previously described high percentage of identified axSpA patients in primary care patients with CLBP. This finding emphasizes the need to introduce a simple referral strategy that can assist primary care physicians in the identification of patients with axSpA who should be referred to specialized care for diagnosis and subsequently for adequate treatment. This is the first study to externally validate a referral rule for axSpA in a primary care CLBP population.
Studying the referral rule performance in an external validation is a valuable step before implementation of the referral rule in clinical practice. Many referral rules have been developed, but only few are used in daily practice. An important reason for this discrepancy is the lack of evidence for external validity.[31] Recently several referral strategies for axSpA were published [13–17, 32], but only few have been externally validated.[13, 14] Moreover there is currently no consensus about what the most appropriate referral strategy for axSpA should be. The available referral strategies for axSpA have been developed in a pre-specified CLBP population or in already referred patients, reflected by the high prevalence of axSpA found in those studies. In contract to these studies, our study population consists of unselected primary care CLBP patients. This is the main strength of our study. Our referral rule has been validated in the population wherein the rule will be used. In our study there was no selection bias for including patients and GPs. For GPs no particular inclusion criteria were used, for patients only ICPC code L03 and age between 18 and 45 years were used, no axSpA specific inclusion criteria were required. Using ICPC code L03 comes with the disadvantage that we invited patients we aren’t currently suffering from low back pain. In ICPC code L03 no chronicity is included. This is confirmed by the finding that more than 70% of the non-participating responders didn’t had low back pain anymore (Fig 1).
The yield of our referral rule is important, the PPV of the referral rule is 30.2%. Assuming the prior probability of axSpA in a CLBP patient is 5% [33], this gives our referral rule an advantage. Our PPV is lower than the PPV of other studies [13], but our referral rule is based on clinical parameters alone. In other studies HLA-B27 testing or imaging is included in the referral strategies, which increases the PPV. However in Dutch primary care there is very limited familiarity with interpretation of SIJ imaging, and also the costs for HLA- B27 testing makes implementation of those referral strategies difficult and makes our ‘simple’ referral rule very applicable in Dutch primary care.
Three predictors from the original referral rule, the ASAS IBP questionnaire, a positive family history for SpA and a good reaction to NSAIDs were also found in the current data, and similar to predictors from the SPACE, MASTER and RADAR studies. [12–14] LBP could not be identified as a predictor anymore. In this current study the proportion of LBP ≤ 5 years was 47%, in CaFaSpA 1 only 38%, however this difference should not bias the effect of duration. Combining the validation and development dataset has several vital advantages, i.e. creating more stable predictor effects and more accurate predictions.
For the application of the rule we propose a cut point that is related to a relative high sensitivity (75%) with a lower specificity (58%). We believe that a relative high sensitivity and thus referring many possible axSpA patients is desirable, considering axSpA is a disease where quality of life increases after the start of treatment. [34] A lower specificity comes at the cost of referring CLBP patients who do not have axSpA, creating extra work in rheumatology practices. However taking into account the impact of axSpA on work participation [3], referring a relative small amount of false positive CLBP patients might even be cost-effective.
A point of discussion is that we used the ASAS criteria to define our outcome definition, namely axSpA. Classification and diagnostic criteria serve a different purpose. The difficulty in the field of axSpA is that there are no diagnostic criteria, there are only classification criteria. We believe that classification and diagnostic criteria have a substantial overlap, and that a diagnosis is almost equal to making a classification in an individual patient. [35] Moreover, classification criteria are more stringent than diagnostic criteria which is also illustrated by two cohorts who compared the diagnosis of a rheumatologist to the ASAS criteria. In the SPACE study were 65 patients diagnosed with axSpA or AS by a rheumatologist. Of these 65 patients were only 55 also classified by the ASAS criteria. [12] In the DECLIC study were 425 patients diagnosed as AS or axSpA, of those fulfilled 324 the ASAS criteria. [36] In both studies are the classification criteria more strict than the diagnosis by a rheumatologist. The specificity was high in both studies (SPACE study 95%, DECLIC 87%) so the fear of ‘over diagnosing’ a lot of patients by using the ASAS criteria, is proven not to be true by those two studies. We have chosen the ASAS criteria as outcome to identify patients as axSpA or no axSpA since this criteria are exactly defined and reproducible for readers, while the diagnosis by a rheumatologist is not. The main purpose of this article was to validate a referral strategy for axSpA in primary care, in this setting is a clear outcome definition desirable.
A remarkable finding in our study is the lower HLA-B27 prevalence (6.2%) in our study compared to other studies. [12, 13, 32] This makes a direct comparison between our study and others difficult. However, the HLA-B27 prevalence was comparable to our first large study in unselected CLBP patients [7] and to the study of Underwood [33], also performed in primary care CLBP patients. There is no evidence that HLA-B27 prevalence is higher among CLBP patients. Therefore we believe that the HLA-B27 prevalence in our study population marks the fact that we did not select on predefined axSpA features and that our referral rule is applicable in and generalizable to all primary care CLBP patients.
In conclusion we provide a stable and robust referral rule that may be applicable as a screening tool in primary care. The next step in the implementation of the referral rule will be, to investigate the clinical impact on GPs behaviour and patients’ outcomes.
Supporting Information
S1 Table. Description of Characteristics for Red flags of (sub)acute low back pain in 579 primary care chronic low back patients.
https://doi.org/10.1371/journal.pone.0131963.s001
(DOCX)
Acknowledgments
We gratefully thank all GPs and participants of this study. We thank our research nurse Esther Hofland, the participating radiologists, Helen Wieringa and Ferry Nobrega, the staff at Diagnostic Centre Schiedam and Voorschoten and the staff of research centre TDIOR BV.
Author Contributions
Conceived and designed the experiments: LH YV JL JH AW. Performed the experiments: LH PB AW. Analyzed the data: LH YV JL JH AW. Contributed reagents/materials/analysis tools: LH YV AW. Wrote the paper: LH YV PB JL JH AW.
References
- 1. Sieper J, Rudwaleit M, Baraliakos X, Brandt J, Braun J, Burgos-Vargas R, et al. The Assessment of SpondyloArthritis international Society (ASAS) handbook: a guide to assess spondyloarthritis. Ann Rheum Dis. 2009;68 Suppl 2:ii1–44. Epub 2009/05/14. 68/Suppl_2/ii1 [pii] pmid:19433414.
- 2. van der Horst-Bruinsma IE, Nurmohamed MT, Landewe RB. Comorbidities in patients with spondyloarthritis. Rheum Dis Clin North Am. 2012;38(3):523–38. pmid:23083753.
- 3. Boonen A, van der Linden SM. The burden of ankylosing spondylitis. J Rheumatol Suppl. 2006;78:4–11. pmid:17042055.
- 4. Callhoff J, Sieper J, Weiss A, Zink A, Listing J. Efficacy of TNFalpha blockers in patients with ankylosing spondylitis and non-radiographic axial spondyloarthritis: a meta-analysis. Ann Rheum Dis. 2014. pmid:24718959.
- 5. Sieper J, van der Heijde D, Dougados M, Mease PJ, Maksymowych WP, Brown MA, et al. Efficacy and safety of adalimumab in patients with non-radiographic axial spondyloarthritis: results of a randomised placebo-controlled trial (ABILITY-1). Ann Rheum Dis. 2013;72(6):815–22. Epub 2012/07/10. pmid:22772328; PubMed Central PMCID: PMC3664374.
- 6. Feldtkeller E, Khan MA, van der Heijde D, van der Linden S, Braun J. Age at disease onset and diagnosis delay in HLA-B27 negative vs. positive patients with ankylosing spondylitis. Rheumatol Int. 2003;23(2):61–6. Epub 2003/03/14. pmid:12634937.
- 7. van Hoeven L, Luime J, Han H, Vergouwe Y, Weel A. Identifying axial spondyloarthritis in dutch primary care patients, ages 20–45 years, with chronic low back pain. Arthritis Care Res (Hoboken). 2014;66(3):446–53. pmid:24072719.
- 8. Hoy D, March L, Brooks P, Blyth F, Woolf A, Bain C, et al. The global burden of low back pain: estimates from the Global Burden of Disease 2010 study. Ann Rheum Dis. 2014;73(6):968–74. pmid:24665116.
- 9. Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2163–96. pmid:23245607.
- 10. Freburger JK, Holmes GM, Agans RP, Jackman AM, Darter JD, Wallace AS, et al. The rising prevalence of chronic low back pain. Arch Intern Med. 2009;169(3):251–8. Epub 2009/02/11. 169/3/251 [pii] pmid:19204216.
- 11. Koes BW, van Tulder M, Lin CW, Macedo LG, McAuley J, Maher C. An updated overview of clinical guidelines for the management of non-specific low back pain in primary care. Eur Spine J. 2010;19(12):2075–94. Epub 2010/07/06. pmid:20602122; PubMed Central PMCID: PMC2997201.
- 12. van den Berg R, de Hooge M, van Gaalen F, Reijnierse M, Huizinga T, van der Heijde D. Percentage of patients with spondyloarthritis in patients referred because of chronic back pain and performance of classification criteria: experience from the Spondyloarthritis Caught Early (SPACE) cohort. Rheumatology (Oxford). 2013. Epub 2013/05/08. pmid:23650623.
- 13. Sieper J, Srinivasan S, Zamani O, Mielants H, Choquette D, Pavelka K, et al. Comparison of two referral strategies for diagnosis of axial spondyloarthritis: the Recognising and Diagnosing Ankylosing Spondylitis Reliably (RADAR) study. Ann Rheum Dis. 2012. Epub 2012/10/16. annrheumdis-2012-201777 [pii] pmid:23065731.
- 14. Poddubnyy D, Vahldiek J, Spiller I, Buss B, Listing J, Rudwaleit M, et al. Evaluation of 2 screening strategies for early identification of patients with axial spondyloarthritis in primary care. J Rheumatol. 2011;38(11):2452–60. Epub 2011/09/17. jrheum.110070 [pii] pmid:21921100.
- 15. Brandt HC, Spiller I, Song IH, Vahldiek JL, Rudwaleit M, Sieper J. Performance of referral recommendations in patients with chronic back pain and suspected axial spondyloarthritis. Ann Rheum Dis. 2007;66(11):1479–84. Epub 2007/04/26. ard.2006.068734 [pii] pmid:17456526; PubMed Central PMCID: PMC2111623.
- 16. Hermann J, Giessauf H, Schaffler G, Ofner P, Graninger W. Early spondyloarthritis: usefulness of clinical screening. Rheumatology (Oxford). 2009;48(7):812–6. pmid:19447774.
- 17. Braun J, Inman R. Clinical significance of inflammatory back pain for diagnosis and screening of patients with axial spondyloarthritis. Ann Rheum Dis. 2010;69(7):1264–8. Epub 2010/06/23. 69/7/1264 [pii] pmid:20566619.
- 18. Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–94. Epub 2009/02/12. pmid:19208371.
- 19. Gebel RS. Semi-automatic coding with ICPC: the Thesaurus, the algorithm and the Dutch subtitles. Stud Health Technol Inform. 1997;43 Pt A:421–5. Epub 1996/12/08. pmid:10179584.
- 20. Garrett S, Jenkinson T, Kennedy LG, Whitelock H, Gaisford P, Calin A. A new approach to defining disease status in ankylosing spondylitis: the Bath Ankylosing Spondylitis Disease Activity Index. J Rheumatol. 1994;21(12):2286–91. Epub 1994/12/01. pmid:7699630.
- 21. van der Heijde D, Lie E, Kvien TK, Sieper J, Van den Bosch F, Listing J, et al. ASDAS, a highly discriminatory ASAS-endorsed disease activity score in patients with ankylosing spondylitis. Ann Rheum Dis. 2009;68(12):1811–8. pmid:19060001
- 22. Roland M, Morris R. A study of the natural history of back pain. Part I: development of a reliable and sensitive measure of disability in low-back pain. Spine (Phila Pa 1976). 1983;8(2):141–4. Epub 1983/03/01. pmid:6222486.
- 23. Krismer M, van Tulder M, Low Back Pain Group of the B, Joint Health Strategies for Europe P. Strategies for prevention and management of musculoskeletal conditions. Low back pain (non-specific). Best Pract Res Clin Rheumatol. 2007;21(1):77–91. Epub 2007/03/14. pmid:17350545.
- 24. Goie The HS, Steven MM, van der Linden SM, Cats A. Evaluation of diagnostic criteria for ankylosing spondylitis: a comparison of the Rome, New York and modified New York criteria in patients with a positive clinical history screening test for ankylosing spondylitis. Br J Rheumatol. 1985;24(3):242–9. Epub 1985/08/01. pmid:3160423.
- 25. Rudwaleit M, van der Heijde D, Landewe R, Listing J, Akkoc N, Brandt J, et al. The development of Assessment of SpondyloArthritis international Society classification criteria for axial spondyloarthritis (part II): validation and final selection. Ann Rheum Dis. 2009;68(6):777–83. Epub 2009/03/20. ard.2009.108233 [pii] pmid:19297344.
- 26. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6):515–24. Epub 1999/03/13. pmid:10075620.
- 27. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38. pmid:20010215.
- 28. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. Epub 1982/04/01. pmid:7063747.
- 29. Balmana J, Stockwell DH, Steyerberg EW, Stoffel EM, Deffenbaugh AM, Reid JE, et al. Prediction of MLH1 and MSH2 mutations in Lynch syndrome. JAMA. 2006;296(12):1469–78. pmid:17003395.
- 30. Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study. BMC Med Res Methodol. 2010;10:7. Epub 2010/01/21. pmid:20085642; PubMed Central PMCID: PMC2824146.
- 31. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. pmid:23393430.
- 32. Braun A, Gnann H, Saracbasi E, Grifka J, Kiltz U, Letschert K, et al. Optimizing the identification of patients with axial spondyloarthritis in primary care—the case for a two-step strategy combining the most relevant clinical items with HLA B27. Rheumatology (Oxford). 2013;52(8):1418–24. pmid:23559574.
- 33. Underwood MR, Dawes P. Inflammatory back pain in primary care. Br J Rheumatol. 1995;34(11):1074–7. Epub 1995/11/01. pmid:8542211.
- 34. Sieper J, van der Heijde D, Dougados M, Mease PJ, Maksymowych WP, Brown MA, et al. Efficacy and safety of adalimumab in patients with non-radiographic axial spondyloarthritis: results of a randomised placebo-controlled trial (ABILITY-1). Ann Rheum Dis. 2012. Epub 2012/07/10. pmid:22772328.
- 35. Yazici H. Diagnostic versus classification criteria—a continuum. Bull NYU Hosp Jt Dis. 2009;67(2):206–8. pmid:19583555.
- 36. Molto A, Paternotte S, van der Heijde D, Claudepierre P, Rudwaleit M, Dougados M. Evaluation of the validity of the different arms of the ASAS set of criteria for axial spondyloarthritis and description of the different imaging abnormalities suggestive of spondyloarthritis: data from the DESIR cohort. Ann Rheum Dis. 2014. pmid:24389295.