Abstract
Objective. In rheumatology, magnetic resonance imaging (MRI) is predominantly applied in the assessment and outcome measurement of rheumatoid arthritis (RA) in hands and wrists, leading to the development of the RAMRIS (RA-MRI-Scoring) system. It was initiated by the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT). The RAMRIS system has not been applied widely in the measurement of feet. We investigated the interreader and intrareader agreement of the RAMRIS scoring system in the assessment of feet in RA.
Methods. Twenty-nine patients with RA who had radiological damage and/or arthritis underwent MRI. Two experienced readers independently read both complete sets. One reader read 6 random sets after the initial session, in order to assess the intrareader agreement. For evaluation of the intrareader and interreader reliability, quadratic-weighted κ scores were calculated per joint and lesion.
Results. For the forefeet, interreader scores were excellent, ranging from 0.77 (bone edema) to 0.95 (bone erosion). Hindfoot interreader agreement scores were highest for erosion (0.90) and synovitis global score (0.88), but edema and synovial thickness agreement were also acceptable (0.83 and 0.86). Intrareader scores were on the whole slightly lower, but excellent.
Conclusion. Reliability (interreader and intrareader agreement) in the assessment of the rheumatoid foot according to the RAMRIS method is excellent.
There is increasing interest in the use of magnetic resonance imaging (MRI) in the diagnosis and monitoring of rheumatoid arthritis (RA)1,2,3. The advantages of MRI over radiography, apart from the absence of ionizing radiation, are the superior imaging of the tissues involved in RA, such as synovial tissue, tendons, sheaths, ligaments, bone, and cartilage4,5,6,7. MRI has proven to be a sensitive and reliable instrument for the detection of inflammatory and destructive changes in RA. Synovial enhancement on MRI closely correlates with the histopathological findings of synovitis8,9, and bone marrow edema represents inflammatory infiltrates or osteitis10,11. In rheumatology, MRI is predominantly applied in the assessments and outcome measurement of RA in hands and wrists, because of their frequent involvement in RA (including early RA) and the fact that these joints are included in traditional clinical and radiological scoring systems in RA12. This has led to the development of the RAMRIS (RA-MRI-Scoring) system, initiated by OMERACT (Outcome Measures in Rheumatoid Arthritis Clinical Trials), and allowing semiquantitative, standardized assessments of inflammatory and destructive changes in RA12,13,14,15,16. There have been some interesting studies in the field of foot MRI in RA17,18, especially the study by Mundwiler, et al, who calculated the predictive value of MRI lesions on the occurrence of radiological damage19. Ostendorf, et al applied the RAMRIS system to the feet18. We examined the interreader and intrareader agreement of the RAMRIS system in the assessment of feet in RA, which to our knowledge has not been done yet.
MATERIALS AND METHODS
Twenty-nine patients with RA from the Arthritis Centre Twente, meeting the 1987 American College of Rheumatology criteria, participated in our study. To be included, patients had foot complaints attributed to arthritis and/or structural damage as a consequence of RA. MRI was performed in both feet and ankles. The ethics committee approved our study and written informed consent was obtained from each patient.
Two readers experienced in the field of musculoskeletal MRI did the scoring. They independently read both complete sets of images after 2 combined sessions of practicing the RAMRIS system on MRI that were not included in our study. One reader read 6 random sets after the initial session, in order to assess the intrareader agreement.
MRI were obtained from a 1.5 Tesla MR scanner (Phillips Medical Systems, Best, The Netherlands) with a 4-element synergy body coil, providing enough coverage for imaging both feet in 1 acquisition. The imaging protocol comprised an axial (short axis) 3-D T1-weighted gradient echo [2 mm slice thickness, 1 mm inplane, TR (relaxation time) 17 ms; TE (echo time) 4.6 ms; flip angle 25], a sagittal T1-weighted SE (spin echo) sequence [3.5 mm slice thickness, gap 0.3 mm; TR 609 ms; TE 19 ms; 3 NSA (number of signal averages)] and a sagittal fat-saturated T2 TSE (3.5 mm slice thickness, gap 0.3 mm; TR 4785 ms; TE 150 ms; 4 NSA). After administration of 15 ml contrast (gadodiamide, 0.5 mmol/ml), a sagittal fat-saturated T1-weighted SE sequence (3.5 mm slice thickness, gap 0.3 mm; TR 609 ms; TE 19 ms; 3 NSA) and an axial (short axis) fat-saturated T1 SE sequence (3.0 mm slice thickness, gap 0.3 mm; TR 609 ms; TE 19 ms; 3 NSA) were acquired. In all sequences, the field of view was 10–14, matrix 256 × 217. Decent images were obtained in all 29 cases.
The MRI sets were scored according to the OMERACT method, a semiquantitative method described by Østergaard, et al12. Bone erosion was defined as a bone defect with sharp margins, visible in 2 planes (when 2 planes were available) with a cortical break seen in at least 1 plane. Bone erosion lesion was scored from 0 to 10 by the volume of the erosion as a proportion of the “assessed bone volume” by 10% increments judged on all available images. For the tarsal bones, the “assessed bone volume” was the whole bone. For long bones, the “assessed bone volume” was from the cortex of the articular surface (or its best estimated position if absent) to a depth of 1 cm. Bone edema was defined as a lesion with ill-defined margins that was neither erosion nor defect and had high signal intensity on T2-weighted sequences. Each bone was scored separately (as for erosions). The scale is 0–3 based on the proportion of bone with edema, as follows: 0, no edema; 1, 1%–33% of bone edematous; 2, 34%–66% of bone edematous; and 3, 67%–100%. This judgment was made on the basis of the preeroded bone, so that maximum erosion scores could not limit the bone edema score. Synovitis was the area in the synovial compartment that shows enhancement of a thickness greater than the width of the joint capsule after gadolinium. Synovitis global score was assessed in the joints of the hindfoot: the tibiotalar joint, the subtalar joint, the talonavicular joint, the calcaneocuboid joint, the tarsometatarsal joint, and the cuneonavicular joint, and in each metatarsophalangeal (MTP) joint. The scale is 0–3. Score 0 is normal, and 1–3 (mild, moderate, severe) are by thirds of the presumed maximum volume of enhancing tissue in the synovial compartment. Synovial thickness was also measured and expressed in exact mm. This was not described in the RAMRIS protocol. The synovial thickness was measured, using electronic calipers, on the point where the enhanced synovium was maximal, in 2 directions, and then averaged.
All MTP joints and joints of the hindfoot were then judged for erosion and edema. For the MTP joints, erosion and edema were scored separately in the distal and proximal part of the joint. The 5 metatarsal bones were scored at the bases for erosion and edema, as were the following tarsal bones: navicular, cuboid, the 3 cuneiforms, talus, and calcaneus. All items per joint were added, and then the interreader and intrareader agreements per joint were calculated.
The data were analyzed individually by joint and lesion to determine how agreement differed by joint and by lesion, and as aggregated scores. Descriptive statistics of each lesion (mean, minimum, maximum, SD, median, 25th and 75th percentiles) for individual joints and aggregated scores were calculated by reader and across both readers. For evaluation of the intrareader and interreader reliability, quadratic-weighted κ scores were calculated per joint and lesion.
The statistical programs used were SPSS 17.0, Analyse-it, and Graphpad Prism.
RESULTS
Demographics and patient characteristics are given in Table 1. Descriptive statistics of the items, including the maximum possible scores as well as the maximum scored range, as scored by each reader, are given in Table 2. Mean and range were presented for both readers. The summed scores of both feet were scored in the full range only for the synovitis global scores. Scores for bone edema were in the lower segment of the range (floor effect).
Synovitis scores and synovial thickness were highest in MTP 1, then in MTP 5 (Figure 1). Both synovitis and synovial thickness were lowest in MTP 4. Bone erosion, both proximal and distal, followed the same patterns. Proximal bone edema was again highest in MTP 1. Distal bone edema was equally distributed between all MTP. Erosion and edema scores were highest in the proximal part of the MTP joint. In the hindfoot, synovitis global score and synovial thickness were highest in the tarsometatarsal joint, followed in descending order by the subtalar joint, the tibiotalar joint, the talonavicular joint, the calcaneocuboid joint, and the cuneonavicular joint. Erosion scores in the hindfoot were highest in the navicular bone and the cuneiform bones, and lowest in the talus and calcaneus (Figure 2). Edema scores in the same region were highest in the talus and calcaneus as well as the cuneiform bones, and lowest in the cuboid bone (Figure 3). Metatarsal erosion scores were highest in MT 2, followed by MT 1, 3, 4, and 5. The bone marrow edema scores were the opposite, with highest scores in MT 5 and 4 and lowest scores in MT 2.
Table 3 shows the interreader and intrareader weighted κ scores of synovitis, synovial thickness, bone erosion, and bone edema, in both the forefeet and hindfeet areas. The interreader scores ranged from 0.77 (bone edema) to 0.95 (bone erosion). The intrareader scores ranged from 0.67 for bone edema to 0.90 for bone erosion. The weighted κ scores for synovitis were higher in the forefeet than in the hindfeet, for both interreader and intrareader agreement. For synovial thickness, on the other hand, agreement was comparable for forefeet and hindfeet, but on the whole lower than the synovitis semiquantitative scores.
DISCUSSION
Our study revealed a good to excellent interreader as well as intrareader reliability for MRI of the rheumatic foot using the RAMRIS system.
In the forefoot, synovitis global score and bone erosion showed excellent weighted κ scores (0.94 and 0.95, respectively), while bone edema κ scores were 0.77 and 0.78. The smaller range of these scores might partially cause the slightly lower κ values for edema. Hindfoot interreader agreement scores were again highest for erosion (0.90) and synovitis global score (0.88), but edema and synovial thickness agreement was also excellent (0.83 and 0.86). As an alternative to the synovial global score, we measured synovial thickness as well. The agreement of this item was in both the foot and the hindfoot joints slightly inferior to the synovial global score. This confirms earlier findings of Lassere, et al, who also found a marginally inferior inter-class correlation coefficient (ICC) score for synovial thickness21. This might be due to the fact that the margins of measurement are not always easy to determine, and in our study, the lack of precise agreement on the plane/view in which to measure. Moreover, gadolinium-containing contrast agents are such small molecules that they leak rapidly out of synovial capillaries and into the adjacent synovial fluid, obscuring the synovium-effusion interface. In small joints such as the MTP, equilibration can occur in as little as 1 minute post-injection. Thus, synovial thickness measurements are not very accurate. Exact knowledge of synovial thickness is not necessary. For the metatarsal bones, erosion and edema interreader weighted κ scores were lower, but still excellent (0.83).
For the intrareader agreement, all weighted κ values were slightly lower, but the trend was the same. The only items with a weighted κ below 0.7 were bone edema of the MTP and bone edema of the tarsal bones. The other κ values lay between 0.73 and 0.90, which is acceptable. The reason for the lower intrareader values compared to the interreader values might be explained by the fact that there was a substantial delay of about 14 months between the first and the second reading, for personal reasons. As well, only 6 sets were scored for the second time, leading to lower ranges, which could affect the κ values. Still, these intrareader weighted κ values are acceptable.
As there are no previous studies on the intrareader and interreader agreement of the RAMRIS in feet, a direct comparison is not possible. However, there is considerable documentation regarding the RAMRIS in the hands and wrists, thanks to the OMERACT MRI working group16,21,22,23,24,25. Østergaard, et al26 described the first multicenter session, without previous training, and found ICC varying from 0.44 to 0.68 for the synovitis global score of the metacarpophalangeal (MCP). Bone erosion proximal ICC of the MCP varied between 0.39 and 0.80. Among the synovitis scores of the wrist joints, ICC showed a range from 0.50 to 0.64. Østergaard, et al found that joint space narrowing could not be scored reliably, and was thus abandoned in further scoring26. After partial adoption of the scoring system and thorough training of the readers, Lassere, et al showed in RAMRIS exercise 3 that the interreader agreement significantly improved21. Average ICC value in the metacarpal regions increased to 0.95 for synovitis global score. In the wrist region, the ICC values were very high for synovitis, erosion, and edema (0.90–0.94). Our study showed comparable quadratic-weighted κ values, especially for synovitis and erosion. Further comparison between hand and foot scoring is limited by the specific differences. The changes in MTP 1 are often degenerative, and not a disease-specific feature; erosive changes in the hindfoot might also be degenerative and a consequence of weight-bearing function.
The good results of our cross-sectional study are promising, but do not guarantee good results in longitudinal data. This needs to be confirmed in followup studies. One limitation of our study is the small number of subjects.
Our study shows that the reliability of interreader and intrareader agreement in the assessment of the rheumatoid foot, according to the RAMRIS method, is highly acceptable. The forefoot especially showed excellent reliability. It is a common subject of study and can easily be compared with other imaging techniques.
- Accepted for publication January 13, 2011.