Modified sharp method: Factors influencing reproducibility and variability*,**

https://doi.org/10.1053/sarh.2001.28304Get rights and content

Abstract

Background and Objectives: In rheumatoid arthritis, joint radiography is still the most frequently used instrument to assess the progression of joint damage. Unfortunately, the poor quality of the radiographic scoring methods available has a negative impact on the power in clinical trials. This study focuses on the influence of the following 4 factors on radiographic scores according to van der Heijde's modification of the Sharp method: intraobserver variation, interobserver variation, follow-up time, and number of measurement occasions within a patient series. Methods: One hundred and seventy-two patients in the early stages of rheumatoid arthritis were followed up. During the first 3 years, radiographs of the hands and feet were taken twice yearly and scored by 3 observers. The scoring process was repeated after an additional 3-year period. Correlation coefficients and differences between observers were calculated to define variability. The influence of the 4 factors on variability was studied. Results: One observer assigned a significantly higher score than the other 2, who had been trained together. Interobserver variability decreased as follow-up time increased. Interobserver correlation coefficients became higher, with smaller differences between observers for progression scores than for absolute scores. Increasing the number of measurements within a patient series led to higher scores. Intraobserver correlation coefficients were high, and a training effect occurred when the time between measurements was 1 year, resulting in lower scores. Conclusions: This study demonstrates that, and shows how, the investigated factors influence the variability of the modified Sharp method. It is extremely important to take interobserver variation into account when designing protocols for multicenter clinical trials. A progression scoring method is recommended for studies assessing radiographic damage or clinical trials. Semin Arthritis Rheum 31:176-190. Copyright © 2001 by W.B. Saunders Company

Section snippets

Patients

Three hundred and seventy-six patients were monitored from 1985 at the outpatient clinic of the Department of Rheumatology at the University Hospital of Nijmegen. Only patients who met the American Rheumatism Association (ARA) classification criteria for rheumatoid arthritis with a disease duration of less than 1 year and who had not been treated with disease-modifying antirheumatic drugs (DMARDs) were included in the study. The follow-up period varied from 0 (only baseline) to 108 months.

I. Reproducibility: Interobserver variation study

Table 1 shows the Pearson correlation coefficients of the absolute Sharp scores between the 3 observers.The correlation coefficients between observers 1 and 2 vary between 0.82 and 0.94; the correlations with observer 3 are much lower—between 0.58 and 0.85.

II. Reproducibility: Interobserver variation, influence of follow-up time

There appears to be higher reproducibility when the follow-up time increases; after 3 years the correlation coefficients between observers 1 and 3 have almost significantly increased (z score = −1.70, P =.089) and those between observers 2

I. Reproducibility: Interobserver variation study

Interobserver variability of radiologic assessment is reasonably well documented. Tables 5 and 6 give an overview of studies in which interobserver variation has been calculated.

Table 5 summarizes the results of 9 studies on interobserver correlation coefficients. It includes only studies using the (modified) Sharp method, possibly with minimal methodologic deviations. In 6 of the studies 6, 8, 9, 10, 11, 12, the correlation coefficients between observers are very high, above 0.90. But in 3

Conclusions

This study demonstrates that interobserver and intraobserver variability, the time interval between readings, and the number of measurements within a given period considerably affect precision, sensitivity to change, and the validity of van der Heijde's modification of the Sharp method. Attention must be paid to these factors if radiographic scoring is to be applied in clinical research. The regular calibration of radiographic scoring (by the same observer and by various observers) seems

Acknowledgements

This study was supported by a grant from The Dutch Arthritis Association.

References (20)

  • DM van der Heijde et al.

    Effects of hydroxychloroquine and sulphasalazine on progression of joint damage in rheumatoid arthritis

    The Lancet

    (1989)
  • JF. Fries

    Toward an understanding of patient outcome measurement

    Arthritis Rheum

    (1983)
  • JT Sharp et al.

    Methods of scoring the progression of radiologic changes in rheumatoid arthritis

    Arthritis Rheum

    (1971)
  • JT Sharp et al.

    How many joints in the hands and wrists should be included in a score of radiologic abnormalities used to assess rheumatoid arthritis?

    Arthritis Rheum

    (1985)
  • A Larsen et al.

    Standardized radiological evaluation of rheumatoid arthritis in therapeutic trials.

  • P Tugwell et al.

    A methodological framework for developing and selecting endpoints in clinical trials

    J Rheumatol

    (1982)
  • CJ Haagsma et al.

    Combination of sulphasalazine and methotrexate versus the single components in early rheumatoid arthritis: a randomized, controlled, double-blind, 52 week clinical trial

    Br J Rheumatol

    (1997)
  • MJ Plant et al.

    Measurement and prediction of radiological progression in early rheumatoid arthritis

    J Rheumatol

    (1994)
  • DMFM van der Heijde et al.

    Prognostic factors for radiographic damage and physical disability in early rheumatoid arthritis. A prospective follow-up study of 147 patients

    Br J Rheumatol

    (1992)
  • EP Nance et al.

    Observer variation in quantitative assessment of rheumatoid arthritis. Part I. Scoring erosions and joint space narrowing

    Invest Radiol

    (1986)
There are more references available in the full text version of this article.

Cited by (22)

  • Imaging as a follow-up tool in clinical trials and clinical practice

    2008, Best Practice and Research: Clinical Rheumatology
    Citation Excerpt :

    Although the Sharp score may have an advantage over the Larsen score in early arthritis37, there is still no universally accepted technique, and modifications to the existing schemes are often proposed. Considerable interobserver variation must be taken into account, especially when dealing with multicentre trials, when applying any of these methods.38 Recently, methods have been proposed for fully automated measurement of joint space width in hand X-rays, automated erosion spotting, and visualization of pathological deviations from healthy bone texture caused by RA.39,40

View all citing articles on Scopus
*

Hilde L. Swinkels, MSc: Health Scientist, Department of Rheumatology, University Medical Centre I Nijmegen, The Netherlands; Roland F.J.M. Laan, MD, PhD: Department of Rheumatology, University Medical Centre Nijmegen, The Netherlands; Martin A. van 't Hof, PhD: Department of Medical Statistics, University Medical Centre Nijmegen, The Netherlands; Désirée M.F.M. van der Heijde, MD, PhD: Department of Rheumatology, University Hospital of Maastricht, The Netherlands; Niek de Vries, MD, PhD: Department of Rheumatology and Clinical Immunology, University Medical Centre Utrecht, The Netherlands; Piet L.C.M. van Riel, MD, PhD: Professor of Rheumatology, Department of Rheumatology, University Medical Centre Nijmegen, The Netherlands.

**

Address reprints to Prof. Dr. Piet L.C.M. van Riel, Department of Rheumatology, University Medical Centre Nijmegen, PO Box 9101, 6500 HB Nijmegen, The Netherlands. E-mail: [email protected]

View full text