Article Text
PDF
Abstract
Objective Recently the Diagnostic and Classification Criteria in Vasculitis Study group developed and published new American College of Rheumatology/EULAR classification criteria for giant cell arteritis (GCA). To test robustness in a different clinical setting and inform clinicians on performance in clinical practice, we aim to externally validate them in patients with a suspicion of GCA referred to our GCA fast-track clinic.
Methods Patients with suspected GCA from the Hospital Group Twente Early GCA in Twente prospective cohort were included. The clinical diagnosis of GCA verified after 6 months of follow-up made by the treating rheumatologist was used as a reference standard. A cut-off score of ≥6 was tested as described in the original article. Area under the receiver operating characteristics curve, sensitivity and specificity were calculated.
Results In total, 133 patients with suspected GCA were included, of whom 53 were diagnosed with GCA and 80 patients were not diagnosed with GCA. The area under the curve (AUC) was 0.96 (95% CI 0.92 to 0.98). Using the proposed cut-off score of≥6, we found that sensitivity was 98.0% (95% CI 89.9% to 100%) and specificity was 57.5% (95% CI 45.9% to 68.5%). The majority of misclassified patients without GCA had classification scores of 6 and 7 as clinical and/or laboratory criteria were often present in our non-GCA population.
Conclusion Our results showed an excellent AUC and sensitivity with a moderate specificity for classification of GCA patients. Considering our relevant study population, we found that the new classification criteria might also be useful for diagnostic purposes, albeit with careful interpretation.
- Giant Cell Arteritis
- Systemic vasculitis
- Epidemiology
Data availability statement
Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author (MvN) upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Recently the Diagnostic and Classification Criteria in Vasculitis Study (DCVAS) group developed and published highly anticipated updated American College of Rheumatology (ACR)/EULAR classification criteria for giant cell arteritis (GCA).
The DCVAS succeeded for the classification criteria to closely represent the current diagnostic process by including imaging modalities and subtype classification.
WHAT THIS STUDY ADDS
The 2022 ACR/EULAR GCA classification criteria had a high sensitivity and a moderate specificity in patients with suspected GCA in a non-academic hospital with a GCA fast-track clinic in this external validation study.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Considering our relevant study population of patients with suspected GCA, the 2022 ACR/EULAR GCA classification criteria might be useful for diagnostic purposes with careful interpretation.
When using the criteria for diagnostic purposes, specificity needs to optimised to prevent unnecessary glucocorticoid treatment and could be improved by adjusting the cut-off value to ≥7 in our cohort.
Introduction
Recently the Diagnostic and Classification Criteria in Vasculitis Study (DCVAS) group developed and published new American College of Rheumatology (ACR)/(EULAR) classification criteria for giant cell arteritis (GCA).1 As the ACR 1990 criteria are outdated, an update was highly anticipated to better reflect current clinical practice regarding GCA and to incorporate increasingly used diagnostic modalities, especially ultrasound.2 3 Also, in the ACR 1990 criteria, focus was mainly on the cranial features of GCA (cranial giant cell arteritis (C-GCA)). Using the 2022 ACR/EULAR classification criteria, we found that patients with a large-vessel giant cell arteritis (LV-GCA) subtype may be better identified, a subtype that has been increasingly acknowledged in the last decade.4 In comparison to the ACR 1990 criteria, these new classification criteria showed higher sensitivity with comparable specificity to identify cases of GCA when internally validated.1
The DCVAS succeeded for the classification criteria to closely represent the current diagnostic process by including imaging modalities and subtype classification. This may imply use of these criteria in a diagnostic setting despite the aim of using it for classification purposes. For the 2010 rheumatoid arthritis ACR/EULAR classification criteria, the scientific community was encouraged to evaluate the diagnostic performance of these criteria.5 Such a proposition was not discussed in the original 2022 ACR/EULAR classification criteria publication. However, informing clinicians on the strengths and drawbacks of these classification criteria when applied for diagnosis is important.
Initial validation of the 2022 ACR/EULAR classification criteria was done in a subset of the large multicentre DCVAS cohort with good results.1 To test robustness of the 2022 ACR/EULAR classification criteria in a different clinical setting and to inform clinicians on the performance in clinical practice, we aim to externally validate them in a cohort of patients with a suspicion of GCA referred to our GCA fast-track clinic (FTC) in a non-academic hospital.
Methods
Design and subjects
The Hospital Group Twente (ZGT) Early GCA in Twente (GET) prospective cohort study is part of the GCA FTC of the rheumatology outpatient department. The FTC is set to evaluate and diagnose patients suspected of GCA within 24 hours of referral. For the ZGT GET cohort study, data are collected during the diagnostic process. Patients diagnosed with GCA are followed up for 5 years to register clinical outcomes. Inclusion of patients suspected of GCA in the ZGT GET cohort started on 1 October 2019 and is still ongoing. For this validation study, patients ≥50 years old with at least 6 months of follow-up were included for analysis.
Data collection
In the ZGT GET cohort, clinical and diagnostic data including patient characteristics, signs and symptoms, laboratory values, imaging (ultrasound, fluorodeoxyglucose–positron emission tomography/computed tomography (FDG-PET/CT) and/or Magnetic Resonance Angiography (MRA)) and temporal artery biopsy (TAB) results were collected. As the FTC incorporated protocolised care, variables regarding signs and symptoms were obtained in a standardised manner. Classification criteria were scored similarly as per by the DCVAS study group methods, including their revision.1 When a classification criteria variable was not described, it was considered absent or not performed, and in that case, no points according to the classification were granted. The need for imaging or TAB was left to the discretion of the rheumatologist, following the EULAR 2018 guidelines.6 Castor study management system (Ciwit B.V., The Netherlands, V.2020.2.24) was used for data management. Patient data were collected from electronic health records. The reference standard was defined as a clinical diagnosis of GCA verified after 6 months follow-up made by the treating rheumatologist.7 For those no longer in our care, the last known diagnosis was used.
Statistics
Baseline characteristics are described as mean values with SD for normally distributed continuous variables, median values with IQRs for non-normally distributed variables and numbers with corresponding percentages for categorical variables. A χ2 test, independent t-test or Mann-Whitney U test was used when appropriate to compare independent groups. The mean score for the classification criteria was calculated and the number of patients diagnosed with GCA (GCA+) and patients suspected of but not diagnosed with GCA (GCA−) per classification score was calculated. The area under the curve (AUC) of the receiver operating characteristic (ROC) curve was calculated. A cut-off score of ≥6 was needed for classification as GCA as described in the original article,1 and sensitivity and specificity were calculated. P- values of <0.05 were considered statistically significant. Statistical analyses were carried out in SPSS V.24.
Results
Baseline characteristics
In total, 136 consecutive patients with suspected GCA between October 2019 and January 2022 were included. Three patients were excluded from analyses as they did not meet the inclusion criterion age over 50 years old, making 133 patients eligible for inclusion in the present study. In the study population (n=133), the mean age was 69.8 years (SD 8.8), and 63.9% (n=85) were female. Furthermore, 53 patients (39.8%) were diagnosed with GCA (GCA+), and 80 patients (60.2%) were suspected of but eventually not diagnosed with GCA (GCA−). Follow-up was present for all GCA+ patients. A last observation carried forward approach regarding the reference diagnosis was used for 40 GCA− patients for whom there was no clinical need for 6 months of follow-up. Baseline characteristics are summarised in table 1. The 2022 ACR/EULAR classification criteria clinical variables jaw claudication, scalp tenderness and abnormal examination of the temporal artery, and all GCA-related laboratory, imaging and biopsy findings were more frequently present in GCA+ patients compared with GCA− patients (p<0.05, table 1). In our study population, 31 GCA+ patients had isolated C-GCA; 8 had isolated extracranial LV-GCA; and 14 patients had both cranial and extracranial GCAs (C-GCA/LV-GCA).
Baseline characteristics
Validity of the 2022 ACR/EULAR classification criteria
The 2022 ACR/EULAR classification criteria had an AUC of 0.96 (95% CI 0.92 to 0.98) (see online supplemental figure S1 for the ROC curve). In table 2, the 2022 ACR/EULAR classification criteria are described for GCA+ and GCA− patients, stratified by the proposed cut-off score of ≥6. Using this proposed cut-off score, we found that sensitivity was 98.0% (95% CI 89.9% to 100%) and specificity was 57.5% (95% CI 45.9% to 68.5%), with a positive predictive value of 60.5% (95% CI 54.2% to 66.4%) and a negative predictive value of 97.9% (95% CI 86.7% to 99.7%). Using this cut-off score, we found that 34 of 80 GCA− patients were not correctly classified (ie, score ≥6). Furthermore, 1 of 53 GCA+ patients had a score of <6 and was therefore not correctly classified. This patient was diagnosed with isolated extracranial large vessel GCA visualised on PET/CT (no ultrasound or TAB performed) without the presence of cranial clinical criteria that are used in the classification model. The mean score for the classification criteria was 13.8 (SD 3.9) in correctly classified GCA+ patients and 3.9 (SD 1.3) in correctly classified GCA− patients. This was 5.0 in the misclassified GCA+ patient (ie, score <6) and 7.6 (SD 1.9) in misclassified GCA− patients (ie, score ≥6). Figure 1 shows the distribution of GCA+ and GCA− patients over all individual classification scores. The majority of misclassified GCA− patients had classification scores of 6 and 7 as clinical and/or laboratory criteria were often present in our GCA− population. To explore possible improvement of the criteria in our population, we re-evaluated the cut-off value, and when using a cut-off value of ≥7, sensitivity was comparable (96.2%), while specificity increased (71.3%).
Supplemental material
Number of patients with suspected GCA for each 2022 ACR/EULAR classification score in our prospective cohort stratified by patients without GCA (GCA−, blue) and patients with GCA (GCA+, orange). Red square indicates not being classified as GCA according to the 2022 ACR/EULAR classification criteria score (<6). ACR, American College of Rheumatology; GCA, giant cell arteritis; GCA+, patients diagnosed with giant cell arteritis; GCA−, patients suspected of but not diagnosed with giant cell arteritis.
2022 ACR/EULAR classification criteria in GCA+ and GCA− patients as diagnosed by the treating physician stratified by a cut-off score of ≥6
Discussion
In this study, we externally validated the 2022 ACR/EULAR classification criteria for GCA. Our results showed an excellent AUC and sensitivity with a relatively low specificity for classification of patients with GCA in the prospective ZGT GET cohort of patients with suspected GCA in our FTC.
Compared with the previous versions of the GCA classification criteria, the 2022 ACR/EULAR classification criteria have been improved.1 The ACR 1990 criteria requires fulfilment of at least three out of the following five criteria for GCA classification: age ≥50 years, new headache, temporal artery abnormality, elevated erythrocyte sedimentation rate (ESR) of ≥50 mm/hour and/or a positive TAB.2 As only three criteria from the ACR 1990 criteria are available, of which one is comparable in both GCA+ and GCA− patients, it is not feasible to apply them in our cohort. Age ≥50 was a requirement for inclusion in this study; hence, all GCA+ and GCA− patients already fulfilled the criterion regarding age. Furthermore, imaging modalities are increasingly used in GCA diagnosis and TAB is not always performed in clinical practice, following EULAR 2018 guidelines.6 In our cohort, only 11 patients (8 GCA+ patients) had TAB performed, and no other diagnostic tools in addition to TAB are incorporated in the ACR 1990 criteria. Our results are in line with the findings of our Spanish colleagues recently published in this journal.8
The updated classification criteria better reflect the increasing use of imaging modalities for GCA diagnosis, of which the importance is addressed before in EULAR recommendations for GCA in clinical practice.6 9 Furthermore, extracranial LV-GCA manifestations are incorporated by including imaging modalities visualising LV-GCA. Nevertheless, diagnosis and classification of patients with LV-GCA are more difficult because these involve atypical signs and symptoms, and therefore these patients are more prone to misclassification as GCA−.9 In the 2022 ACR/EULAR classification criteria internal validation, this was shown by a lower sensitivity for patients with LV-GCA.1 Our results showed that the only misclassified GCA+ patient (using a cut-off score of ≥6) was diagnosed with isolated extracranial LV-GCA. However, since only eight patients with isolated extracranial LV-GCA were included in the ZGT GET cohort, larger numbers of isolated patients with LV-GCA are needed to obtain a better understanding regarding classification of GCA subtypes in an external validation.
Furthermore, the 2022 ACR/EULAR classification criteria are developed to serve a classification rather than a diagnostic purpose. Nevertheless, as GCA is difficult to diagnose and delay in diagnosis is unwanted,10 11 the use of these criteria as a diagnostic aid would serve an unmet need. The low specificity observed in the present study may be partly explained by the control group used in the present study. In the original article, patients older than 18 years who had a condition of any type of vasculitis or vasculitis mimic were included, an issue also recently raised in a comment to the 2022 ACR/EULAR classification criteria.12 Our control group consisted exclusively of patients with suspected GCA seen at the rheumatology outpatient department with most often a new onset of headache and elevated inflammatory markers. Our cohort is therefore more representative to clinical practice regarding the use of diagnostic criteria, including a certain overlap in clinical presentation between the patients with or without GCA. Misdiagnosis using the 2022 ACR/EULAR classification criteria in the GCA− group with similar clinical presentation should therefore be considered. This might give reason to reconsider the cut-off score for diagnostic purposes in a population with suspected GCA, as a score of ≥7 performed with similar sensitivity but higher specificity in our cohort. Because of this low specificity, GCA diagnosis should not be based on the classification criteria but on clinical presentation confirmed with positive imaging or TAB, following EULAR recommendations.6 7
A major strength of this validation study was that we included patients with suspected GCA who participate in a prospective cohort, where data were systematically collected as part of routine care. The criteria set was modified after first publication regarding points for elevated inflammatory parameters, and the updated criteria set was used in this study. The limitations of this study include the small number of patients, inherent to studying a rare disease in a single-centre setting. Furthermore, tongue claudication and neck stiffness were not registered in our database; however, we expect no impact on our results. Both criteria were combined with another associated clinical criterion (jaw claudication or shoulder stiffness) in the score.
In conclusion, the 2022 ACR/EULAR classification criteria performed well in patients with suspected GCA in a non-academic hospital with a GCA FTC. GCA+ patients as diagnosed by the treating physician were better classified using the proposed cut-off score compared with GCA− patients, as only one GCA+ patient was misclassified. However, a higher number of GCA− patients were misclassified as GCA+. This directly translates into the relatively low specificity observed in our cohort. Considering our relevant study population, the new classification criteria might also be useful for diagnostic purposes, although with careful interpretation. When using the 2022 ACR/EULAR classification criteria as a diagnostic aid, specificity could potentially be improved by adjusting the cut-off value to ≥7, which performed better in our cohort. The high sensitivity we observe is essential to avoid misdiagnosis of patients with GCA, considering severe GCA-related complications. Nevertheless, a high specificity, in addition to the current high sensitivity, is needed to avoid starting unnecessary glucocorticoid treatment in clinical practice, warranting further investigation in other large international cohorts.
Data availability statement
Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author (MvN) upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by The METC Twente considered the study as not subject to the Medical Research Involving Human Subjects Act. The participants gave informed consent to participate in the study before taking part. The study was conducted in accordance to the Helsinki code.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @ZGT_info
Contributors All authors read and approved the submitted manuscript. MvN acts as guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The METC Twente considered the study as not subject to the Medical Research Involving Human Subjects Act.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.