Article Text
PDF
Abstract
Objective To evaluate the impact of a standardised training programme including equipment adjustment for experienced musculoskeletal ultrasonographers without previous experience in vascular ultrasound (US) on the reliability of US in the diagnosis of giant cell arteritis (GCA).
Methods In this prospective, non-interventional observational cohort study, patients suspected of GCA were evaluated by US by one of five rheumatologists with long-standing experience in musculoskeletal US (>8 years), trained using a standardised training programme including equipment adjustment. Images of cranial and large vessels were subsequently evaluated first by the performing ultrasonographer and thereafter by a blinded external expert (gold standard).
Results In three Danish centres, 112 patients suspected of GCA were included. According to the external expert, vasculitis changes were seen in 66 patients, in 45 of them with only cranial involvement, in 14 with both cranial and large vessel involvement, while in seven patients isolated large vessel vasculitis was found. The reliability was excellent between the local ultrasonographer and the US expert for the overall GCA diagnosis regarding the diagnosis of cranial and for large vessel GCA, with an interobserver agreement of 95–96%, mean kappa values of 0.88–0.92 (95% CI 0.78 to 0.99). Excellent reliability (mean kappa 0.86–1.00) was also found for the US examination of the individual arteries (temporal, facial, common carotid and axillary).
Conclusion The US training programme resulted in excellent agreement between trainees and an expert in patients suspected of GCA and may thus be applicable for implementation of vascular US in clinical practice.
- Giant Cell Arteritis
- Ultrasonography
- Systemic vasculitis
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
INTRODUCTION
Giant cell arteritis (GCA) is a systemic vasculitis involving large and medium-sized vessels in individuals older than 50 years.1 Early diagnosis and treatment of patients with GCA are important due to the risk of significant complications including blindness and stroke.2 Because of a high level of evidence of good test performance, accessibility, minimal invasiveness, low cost and good overall performance, EULAR recommends vascular ultrasound (US) of the temporal and axillary arteries as the primary imaging test in patients suspected of cranial GCA (cGCA).3
Supplemental material
The EULAR recommendations3 highlight that US examination should be performed by a trained specialist using appropriate equipment, operational procedures and settings. The reliability of US examinations, which has often been a concern, can be improved by specific training. A proposal for the technical and operational parameters has been developed4; nevertheless, no specific training programmes exist, all the studies in a systematic literature review being conducted by expert groups.5
Vascular US examinations are regarded as strongly operator-dependent, and to date, very few studies have investigated US reliability. When evaluating preselected images/videos from patients with an established GCA diagnosis, high interobserver and intraobserver agreement has been demonstrated in two studies.6 7 However, only moderate agreements were found in the Ultrasound Compared to Biopsy of Temporal Arteries in the Diagnosis and Treatment of Giant Cell Arteritis (TABUL) study.8 The reliability in real-time patient assessment has been evaluated in one study in patients with an established GCA diagnosis9 and in two studies in patients suspected of GCA, with excellent agreement among sonographers.10 11 In these two studies, only two sonographers from the same centre participated.
In the TABUL study,8 the sonographers were either experts or less experienced sonographers, who had successfully passed a training programme that included a test on individual competence in interpreting US videos and US examinations of 10 healthy persons as well as of one patient with active GCA. This was the first attempt in which a standardised vascular US training programme in the diagnosis of GCA had been evaluated as part of a trial. The diagnostic accuracy of US in this study was significantly lower than that in recently published studies and in the latest meta-analysis.5 These results are probably related to the limited US experience of a significant number of sonographers, as documented by a 17% increase in US sensitivity for GCA diagnosis in US scans performed by operators after having performed >10 scans within the study.8
The primary aim of this study was to evaluate the impact of standardised training programme including equipment adjustments for musculoskeletal experienced ultrasonographers with no experience in vascular US before training initiation on the reliability of US in the diagnosis of GCA. The secondary aim was to assess the agreement on GCA-US diagnosis between the trained GCA-US ultrasonographers and an expert on US in GCA.12
METHODS
Study design and sample size
The study was designed and reported according to the recommendations that are given in the ‘Guidelines for Reporting Reliability and Agreement Studies’ statement.13 The sample size was determined by the design of the main study, at least 100 patients being considered as a representative (and feasible) sample.
Sampling method
This prospective, non-interventional observational cohort study was performed during 3 years (from April 2014 to June 2017) in three Danish centres. At baseline, relevant clinical and paraclinical data were collected. Consecutive patients suspected of having GCA were included in the study using the following criteria: age ≥50 years with signs and symptoms which indicated the presence of GCA: new localised headache, jaw claudication, tenderness and/or reduced pulsation of the temporal artery, scalp tenderness, new-onset visual disturbances (anterior ischaemic optic neuropathy and amaurosis fugax/diplopia), elevated inflammatory parameters without other explanation (C reactive protein (CRP) and/or erythrocyte sedimentation rate) and/or polymyalgic symptoms. The US examination and temporal arterial biopsy were performed within 7 days after diagnostic initiation (ie, enrolment). Patients with previous GCA diagnosis, use of more than 20 mg prednisolone for more than 7 days before the US examination and tissue sampling, long-term use (>1 month) of less than 20 mg prednisolone daily until 3 months before study start, mental disease and/or alcohol or drug abuse that affected the patients’ ability to give informed consent were excluded from the study. All participating patients gave written consent according to the Declaration of Helsinki. The study was conducted according to Good Clinical Practice.
Ultrasound set-up
At inclusion, a vascular scanning of 12 vessels was performed: bilateral evaluation of the facial arteries (FA), common carotid arteries (CA),AA and three sites on the TA (common superficial artery and parietal and frontal branches).
All US examinations in each centre were conducted with the same equipment (Hitachi Preirus/Ascendus in Esbjerg/Silkeborg, GE Logic E9 in Glostrup) and settings were kept unchanged throughout the study (online supplemental file I). Equipment settings, scanning techniques and image analysis were performed according to the International Workshop on Ultrasound in Large Vessel Vasculitis & Polymyalgia Rheumatica standards14 and were in line with the proposal by Terslev et al.4 All arteries were examined in longitudinal and transversal views. In both projections, B-mode and colour Doppler examination was performed. A video clip of at least 3 seconds applying the compression technique15 on the TA and FA was stored. The intima-media thicknesses (IMT) of the CA and AA were measured, and a colour Doppler video clip of at least 3 seconds was stored in an image database.
US images/videos were evaluated (online supplemental file II) by the performing ultrasonographer and subsequently by a rheumatologist (APD) with long-standing experience in GCA US (10 years of experience with over 3000 vascular US examinations of the TA and the large supra-aortic vessels),12 who made the final US diagnosis (gold standard). The ultrasonographers and APD were blinded to all clinical, laboratory and biopsy data. A positive sign for vasculitis in the TA branches and in the FA was defined as a hypoechoic and increased IMT (halo sign) and a positive compression sign. Since the TA and FA have widely variable diameters, a specific IMT as the cut-off for vasculitis was not defined in these vessels. A homogeneous intima-media complex increased thickness in the AA of >1 mm and ≥1.5 mm in the CA was defined as vasculitis.12
Supplemental material
Ultrasonographic training
Five rheumatologists from Denmark with more than 8 years of experience in the use of musculoskeletal US (Doppler included) but with no previous experience in vascular US were trained at the ‘International Workshop on Ultrasound in Large Vessel Vasculitis & Polymyalgia Rheumatica’ held by APD and a group of international experts on vascular US.14 The course contained 5 hours of theoretical training and 10 hours of hands-on US examination of 12 healthy persons and eight patients with GCA pathologies (both cranial and large vessels) under supervision. Four months later, the training programme was followed by a 2-day workshop organised by the investigators in Denmark, with additional training and standardisation of the scanning technique and optimisation of equipment settings supervised by APD. The workshop included 6 hours of supervised hands-on training in four healthy subjects and four patients with GCA with both cranial and large vessel pathologies, followed by 1 hour of image evaluation. From the beginning of the vascular US training (March 2013) to the initiation of the study (April 2014), the five ultrasonographers had performed ≥50 vascular US examinations, with a minimum of 50% scans in hot cases or patients with an established GCA diagnosis (figure 1). APD was available for feedback and supervision of images/videos during that period.
Ultrasound training programme.
GCA, giant cell arteritis; US, ultrasound.
Statistical methods
For the quantification of interobserver agreement and reliability, we used absolute agreement and (unweighted) kappa. Kappa coefficients were interpreted according to Landis and Koch, with κ values of 0.0–0.2 considered poor, 0.2–0.4 fair, 0.4–0.6 moderate, 0.6–0.8 good and 0.8–1 excellent.15 All statistical tests were based on a comparison between the ‘gold standard’ (APD) and each of the five experienced Danish ultrasonographers. The statistical analyses were based on the observed data, and no attempt to perform imputations for missing data in the primary analyses was done, that is, it was assumed that missing data were missing completely at random. All the statistical analyses were performed using SAS V.9.4 (SAS Studio).
RESULTS
Patient characteristics
From three Danish centres, 112 patients suspected of GCA were included as presented in the flow diagram (figure 2). The 112 patients had a mean age of 72.4 years±SD 7.9 years; 66 (59%) were women. The mean duration of symptoms before referral to the hospital was 5.9 weeks±SD 4.43 weeks, with initial mean CRP levels of 69.4±SD 61.5 mg/L, and a mean treatment duration prior to US examination of 0.91 days±SD 1.55 days. The majority of patients (92%) reported a newly emerged localised headache and 43.8% experienced symptoms of polymyalgia rheumatica at the time of debut (table 1).
Study flow diagram.
GCA, giant cell arteritis; US, ultrasound.
Patient characteristics
US findings
In total, 1344 arteries/branches were evaluated by US. According to the external US expert, vasculitis changes were observed in 66 patients, in 45 cases with only cranial involvement, in 14 with both cranial and large vessel involvement, while in seven patients isolated large vessel vasculitis was found (table 1). In the 54 patients with cGCA in whom the compression sign was performed, a positive halo sign was also observed. FA involvement was observed in 23 out of 59 patients with cGCA (table 1). In patients with large vessel GCA (LV-GCA) (n=21), the AA was more often affected (90%) (table 1), and in 14 out of 19 cases, the findings were bilateral, with an IMT range from 1.1 mm to 2.5 mm (mean 1.72 mm±SD 0.47). AC involvement was observed in six patients, with a mean IMT of 1.97 mm±SD 0.44, one of whom exhibited isolated CA involvement (IMT 2.1 mm bilateral) without vasculitis changes in the AA.
Overall reliability and agreement
The reliability between the local ultrasonographers and the US expert was excellent for the final diagnosis of GCA. Of the total number of patients (112 patients), agreement on the final diagnosis was found in 108 patients. In 43 of these patients, the findings were normal, and in the remaining 65 patients, pathological findings were found. The mean agreement was 96%, mean kappa 0.92 (95% CI 0.85 to 099). Furthermore, the agreement was also excellent for the evaluation of cGCA and LV-GCA, kappa 0.89 (95% CI 0.81 to 0.98) in 95% and kappa 0.89 (95% CI 0.78 to 0.99) in 96% (table 2) Analysis of the reliability at the vessel level was also performed, evaluating the agreement between the US expert and the local ultrasonographer for the halo sign and the compression sign in the TA/FA and for halo sign in the AA/AC. The reliability was excellent in all vessels for both the halo sign and the compression sign, with an agreement between 94% and 100% and k coefficients from 0.86 to 1.0 (table 2).
Overall inter-rater reliability and agreement
The agreement for the overall diagnosis of GCA, cGCA and LV-GCA for each centre was excellent, with a mean agreement of 96%, 93% and 94%, respectively. The distribution of the US examinations varied between the performing ultrasonographers. Two of the operators performed under 10 US scanning (2 and 8), while the others evaluated 17, 22 and 63 patients. An additional analysis on the agreement between the US expert and the single ultrasonographer was performed for both the final diagnosis and on the vessel level, with excellent agreement (range from 88% to 100%). Furthermore, no differences regarding agreement were observed at the beginning of the study (first five scans of the four sonographers) comparing with the rest of the study.
DISCUSSION
This is the second multicentre study evaluating the impact of a standardised training programme and the only study to date where an extended training was used for musculoskeletal experienced ultrasonographers without previous experience in vascular US for obtaining the diagnosis GCA and for agreeing on vascular pathology. We found excellent US reliability for both the overall GCA diagnoses, but also on a vessel level using both the halo and the compression signs as elementary US vasculitis lesions.
Despite the growing body of evidence supporting the utility of US in GCA, standardised training programmes and their impact on reliability are lacking.3 Previously, one study evaluated a standardised training programme and reliability exercise for rheumatologists without previous experience with vascular US6 and found excellent interobserver agreements in line with our study. However, the impact of a training programme was only evaluated in preselected static images and not in patients in routine care with suspected GCA. In the TABUL study,8 several of the participating ultrasonographers were not familiar with the vascular US and underwent a limited training programme, including a test on the individual competence in interpreting vascular US videos and performing vascular US examinations of 10 healthy persons as well as of one patient with active GCA. Previous US experience was not a prerequisite for participation. Only 53% of the participants passed the examination, only a few (16%) of whom passed the examination in the first attempt. In the TABUL study, the interobserver agreement based on blinded posthoc image analysis by 12 different sonographers was only moderate, illustrating the challenges presented in the education for the vascular US in GCA.
A study from 201215 demonstrated that the sensitivity of US in GCA decreases within the first days of steroid treatment, due to the effect of the treatment on the inflamed vessel wall swelling, with decreasing/disappearance of the halo sign. A recent study from the Outcome Measures in Rheumatology US subgroup on large vessel vasculitis evaluating the reliability of US in patients was performed in two steps.9 The first step was an exercise in patients who turned out to have very little pathology due to long prednisolone treatment and long-standing disease, resulting in inconclusive results. Furthermore, the participants were not familiar with the US equipment used in the exercise. Subsequently, a patient-based exercise was carried out in patients with early disease and shorter duration of prednisolone treatment, with participants having more training and time with the US equipment and its settings. In the first exercise, inter-reader reliabilities were fair to moderate (light k 0.29–0.51), while in the main exercise, the inter-reader reliability was increased significantly (good to excellent, light k 0.76–0.86) for the overall diagnosis of GCA. In our study, the treatment duration of the patients before the US examination was very short (mean 0.91±SD 1.55 days), which may have facilitated the high reliability in vessel pathology in our study. Furthermore, the participants were well acquainted with the US equipment and settings.
In our study, we used halo and compression signs as indicators of vessel wall swelling. The reliability of the compression sign has been tested in one clinical study to date.9 In this study, a reliability evaluation of the compression sign in US of the temporal arteries was performed, comparing the results of an experienced vascular specialist with those of a rheumatologist not familiar with vascular US, who had undergone a short training course with five supervised compression US examinations.9 A high interobserver agreement was found regarding the compression sign in US of temporal arteries (98% agreement), in line with our results. However, the study did not provide information about the training protocol. We are also the first to present results on the reliability of the compression sign for the FA, which is as high as for the TA (table 2)
One limitation of our study is that one single ultrasonographer contributed to approximately half of the examinations (63 out of 112); however, no differences were observed regarding agreement on the ultrasonographer level. Another limitation is the lack of a real control group in the assessment of the impact of the training programme or a comparison of reliabilities before and after initiation of the training programme. In the TABUL study, the agreement with GCA findings was low between the performing ultrasonographer and the expert reviewer of images. In 47 out of 162 cases, the expert did not find any GCA changes on the images and videos evaluated as pathological by the performing ultrasonographer.8 Using the TABUL study as a substitute for a control group, our study indicates that a more extended and structured training programme improves the reliability of vascular US examinations. Furthermore, no changes with regard to agreements were observed at the ultrasonographer level in the first five scans in the study compared with the rest of study, indicating that our training programme was sufficient, with no additional improvement being seen during the study.
CONCLUSION
The training programme together with previous musculoskeletal US experience resulted in excellent US reliability in patients suspected of GCA, both for the overall diagnosis and at the vessel level. The training programme may be used for the implementation of the vascular US in clinical practice.
Key messages
What is already known about this subject?
Vascular ultrasound (US) is increasingly used for confirming or excluding a suspected diagnosis of giant cell arteritis (GCA) in clinical practice.
Vascular US is regarded as strongly operator-dependent, and US reliability for the diagnosis of giant cell arteritis may be improved by specific training.
What does this study add
This is the first multicentre study evaluating the impact of a standardised training programme for ultrasonographers without previous experience in vascular US.
Using a dedicated training programme with theoretical lectures and practical US examination of both healthy controls and patients with GCA pathology under supervision, a high level of reliability was achieved regarding vessel pathology and patient diagnosis.
How might this impact on clinical practice?
Our training programme may be used for the implementation of vascular US in clinical practice.
Acknowledgments
For the purpose of this work, the authors used the technical facilities of OPEN (Open Patient data Explorative Network), Odense University Hospital, Odense, Denmark.
Footnotes
Contributors All authors were involved in drafting the manuscript or revising it critically for important intellectual content, and all authors read and approved the final manuscript. SC, APD, LT, UMD, UF, KL and TL contributed substantially to the conception and design of the study. SC, APD, LT, UMD, UF and TL made substantial contributions to the acquisition of data. SC, LT, UMD, UF and TL performed and scored US examinations; APD scored all US examinations. SC and RC contributed substantially to the analysis and interpretation of data.
Funding The Hospital of Southwest Jutland, Esbjerg, granted research grants for the study as a part of a PhD. The Parker Institute, Bispebjerg, and Frederiksberg Hospital are supported by a core grant from the Oak Foundation (OCAY-18-774-OFIL). None of the funding bodies had any role in the design of the study; in collection, analysis and interpretation of data and in writing the manuscript.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study was approved by The South Denmark Region Committees on Health Research Ethics (reference number S-2014003) and The Danish Data Protection Agency (reference number 2008-58-0035). All patients gave their written informed consent.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The data that support the findings of this study are not publicly available and restrictions apply to the availability of these data according to the Danish Data Protection Regulation. Data are however available from the authors upon reasonable request and with permission of the Southwest Hospital in Esbjerg, legal services of the Research & Innovation Organisation and approval from the Danish Data Protection Agency.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.