Responsiveness of a clinical measure is defined as the ability to detect a clinically meaningful change, such as a change that clinicians or patients think is discernible and important.¹ In a clinical trial, advanced knowledge of instrument responsiveness aids in the selection of measures, permits accurate estimation of sample size to assure adequate statistical power, and assists in prioritising and reducing the number of outcomes to be assessed.²

In recent years, health related quality of life (HRQoL) has been increasingly recognised as one of the core outcome measures for the assessment of children with rheumatic disorders in both clinical practice or in therapeutic trials.^3,4 A number of multidimensional paediatric HRQoL measures incorporating physical, emotional, mental, and social health dimensions have been developed and validated for use in paediatric rheumatic diseases.^5–9

There has been much discussion about the relative strengths of disease specific versus more generic measures of disease outcomes.¹⁰ Generic instruments, such as the HRQoL, enable comparison across conditions and facilitate benchmarking with healthy populations, whereas condition specific instruments may enhance sensitivity for health domains germane to a particular chronic condition. However, little information is available on the relative responsiveness of generic and condition specific measures in childhood rheumatic diseases in general and in juvenile idiopathic arthritis (JIA) specifically.^11,12 Furthermore, to our knowledge the responsiveness of clinical measures in JIA patients who have received intra-articular corticosteroid injections (IAC) has never been studied.

In the present study, we compared the relative responsiveness of the traditional condition specific measures with that of a generic paediatric HRQoL instrument for outcome assessment of IAC injection in a cohort of patients with oligoarticular JIA. We focused our analysis not only on the traditional responsiveness statistics, but also investigated the discriminative ability of each instrument to distinguish patients who improved from those who did not, using the physician’s and the parent’s judgements on the treatment outcome as external criteria.

PATIENTS AND METHODS

Patients

All consecutive patients who met the 1997 International League of Associations for Rheumatology criteria for the diagnosis of JIA,¹³ were aged <18 years, had oligoarthritis (⩽4 involved joints), and received an IAC injection in one or more joints between February 2000 and October 2001 were included in the study. All patients were outpatients; however, the younger patients, who required a general anesthetic to undergo the procedure, were assessed on an inpatient basis. Assessment of measures was performed on all patients before the IAC injection and after 6 months. The 6 month timeframe was chosen because it has been adopted in previous studies on the efficacy of IAC injections in JIA.¹⁴ Patients who received further IAC injections in the study period were excluded. No patient was excluded for other reasons or was lost to follow up.

Measures

Condition specific measures

The instruments used were: physician’s global assessment of overall disease activity (physician global) measured on a 10 cm visual analogue scale (VAS) (0 = no activity; 10 = maximum activity); parent’s global assessment of the child’s overall well being (parent global) on a 10 cm VAS (0 = very good; 10 = very poor); parent’s assessment of the child’s pain (parent pain) on a 10 cm VAS (0 = no pain; 10 = very severe pain); functional ability assessment through the Italian version of the Childhood Health Assessment Questionnaire (CHAQ)¹⁵ (0 = best; 3 = worst); number of joints with active arthritis, defined as swelling or, if no swelling was present, as limitation of movement with either pain upon movement or tenderness; number of swollen joints; number of joints with pain upon movement/tenderness; number of joints with limited range of motion (LROM); Global Articular Severity Score (GASS), calculated as the sum of the severity ratings of swelling, pain/tenderness, and LROM, as previously described;¹⁶ morning stiffness, measured in minutes; erythrocyte sedimentation rate (ESR) (Westergren method); and C-reactive protein (CRP) (nephelometry). The articular indices were assessed in a total of 67 joints (those that are included in the normal clinical evaluation). These variables, as well as the physician global, were assessed in each patient by the same investigator (AR or SV) both at baseline and 6 months. Likewise, the CHAQ, the parent global, and the pain global were completed by the same parent on both assessments. Because in most patients only a subset of the involved joints were injected and the therapeutic effect was expected to occur mostly in the injected joints, the responsiveness analysis for the articular variables was limited to the joints that were injected at baseline.

Generic health status measure

The patient’s quality of life was assessed through the Italian version of the Child Health Questionnaire (CHQ),¹⁵ a generic health instrument designed to capture the physical and psychosocial functioning of children ⩾5 years of age. Parents are instructed to take into consideration the 4 week period preceding their compilation of the questionnaire. The CHQ measures through 50 items/questions the following health concepts: global health, physical functioning, social role, emotional/behavioural limitations, social role/physical limitations, bodily pain discomfort, behaviour, global behaviour, mental health, self esteem, general health perception, change in health, emotional impact on the parent, impact on the parent’s personal time, limitations in family activities, and family cohesion. The CHQ yields two summary scores, the physical score (PhS) and psychosocial score (PsS). The mean (SD) norm based score from cross cultural general populations for both PhS and PsS is 50 (10), with higher scores indicating better health. In each patient, the CHQ was completed by the same parent both at baseline and at 6 months.

External criteria

At the 6-month evaluation, the physician (AR or SV) and the parent were asked to judge for each patient the overall result of the previous IAC injection on a six point scale: complete remission, much improved, slightly improved, no change, slightly worse, much worse (treated as 6, 5, 4, 3, 2, and 1, respectively). Both the physician and the parent were asked to indicate the category of response after having completed their respective 6 month global assessments and were not allowed to review their baseline global assessments. Furthermore, the physician was blinded to the measures of function (particularly the CHAQ and the CHQ), though they were not blinded to the timing of the measures (pre-injection versus post-injection). Although both the physician’s and the parent’s assessments were mostly focused on the specific effect on the injected joints, they were also asked to take into account the more general effect on the patient’s health (morning stiffness, functional ability, general wellbeing). For the purpose of the analyses, patients were classified as clinically improved if they were judged as in complete remission or much improved, and as clinically non-improved if they were classified as slightly improved, unchanged, slightly worse, or much worse; patients were defined as clinically stable if they were judged as slightly improved or unchanged.

Analyses

Responsiveness statistics

Responsiveness statistics included the standardised response mean (SRM), the effect size (ES), and Guyatt’s method.¹⁷ The SRM was calculated as the mean change in score divided by the SD of individuals’ change in score¹⁷ and the ES as the mean change in score divided by the SD of individuals’ baseline score.¹⁸ Guyatt’s method allows calculation of responsiveness by using an external criterion, such as objective or self perceived improvement. In contrast to the SRM and ES, this approach accounts for score variability in apparently stable subjects. It is based on the comparison of signal (mean change in clinically improved patients) with noise (SD of change in stable patients). According to this method, the responsiveness was calculated as the ratio of mean change score in patients who were classified as improved according to the physician’s or parent’s external judgement divided by the SD of change in patients who were classified as stable (i.e. slightly improved or unchanged).¹⁹ This approach is expected to give higher ratios than the SRM and ES because the denominator becomes smaller with the inclusion of the SD of change only in stable subjects. According to Cohen,²⁰ the threshold levels for SRM, ES, and Guyatt’s method were defined as follows: ⩾0.20 = small, ⩾0.50 = moderate, ⩾0.80 = good.

Discriminative ability

To investigate the ability of each instrument to distinguish patients who improved from those who did not, we employed the correlation method and the receiver operating characteristic (ROC) method using the physician’s and the parent’s judgements of the outcome of the IAC injection as external criteria. With the correlation method, the external criterion is correlated with the change in instrument score, with the strength of the correlation being the measure of responsiveness. Correlations were calculated with the Spearman’s correlation coefficient.²¹ The sensitivity and specificity of each measure in identifying the treatment outcome were calculated using standard formulae.²² This information was used to plot a ROC curve for each instrument. Values for sensitivity and for false positive rates (1−specificity) were plotted on the y and the x axes of the curve, respectively. The area under the ROC curve (c curve) was then calculated.^21,23 The c value (range 0–1, with 1 representing the ideal area) was used to rank the ability of each instrument to differentiate improved from nonimproved patients. The 95% confidence intervals (CI) of the c values were also calculated. Threshold levels for the Spearman’s correlation coefficient were arbitrarily defined as: ⩾0.60 = good; ⩾0.50 = moderate; ⩾0.40 = small. Threshold levels for the AUC-ROC were arbitrarily defined based on the lower limit of the 95% CI as: ⩾0.70 = good; ⩾0.60 = moderate; ⩾0.50 = small.

RESULTS

There were 44 consecutive patients with JIA, all with the oligoarticular onset subtype, aged 2.6 to 14.8 years (mean 7.2 years), included in the study. There were nine boys and 35 girls; onset age ranged from 1.1 to 11.6 years (mean 3.7) and the disease duration from 1.2 to 10.4 years (mean 3.4). At baseline, all patients had oligoarthritis (⩽4 affected joints); the mean number of active joints was 2.05 (median 2; range 1 to 4) and the mean number of injected joints was 1.4 (median 1; range 1 to 3). There were 28 patients injected in one joint, 10 in two joints, and 6 in three joints. The joints injected were the knee (50 injections), the ankle (13), the elbow (2), and the wrist (1). During the study period, 24 patients did not take any systemic medication, whereas 20 patients received non-steroidal anti-inflammatory drugs and 8 patients received methotrexate.

Assessment of treatment outcome

At 6 months, the physicians judged 23 patients as improved, 14 as stable, and 7 as worsened, whereas the parents judged 27 patients as improved, 13 as stable, and 4 as worsened. For 34 patients, the physician’s and the parent’s opinion on the treatment outcome was concordant, whereas for 10 patients there was discordance. The mean scores at baseline, the mean score changes between the baseline and the 6 month evaluation, and the mean score changes for the different categories of the physician’s or parent’s judgement of the treatment outcome (improved, stable, or worsened) are reported in table 1.

Responsiveness statistics and discriminative ability

The results of responsiveness statistics and discriminative ability assessments obtained with the use of the physician’s or the parent’s external indicators of change are shown in table 2. All responsiveness statistics and the examination of the discriminative ability consistently ranked the physician global as the most responsive measure. The parent global and the parent pain revealed a poorer ability both in measuring baseline versus 6 month change and, to a lesser extent, in distinguishing patients who improved from those who did not. All articular variables yielded satisfactory results with the traditional responsiveness statistics and revealed good discriminative ability, although the painful and limited joint count appeared to be less responsive than the active and swollen joint counts and the global articular severity score. The CHQ-PsS proved more responsive in detecting clinical change than the CHAQ and the CHI-PhS; both summary scales of the CHQ showed better ability in discriminating responders versus nonresponders compared with the CHAQ. The morning stiffness revealed fair responsiveness, but poor discriminative ability, whereas the ESR and CRP revealed both poor responsiveness and discriminative ability. A summary of the results of responsiveness statistics and discriminative ability assessments is presented in table 3.

DISCUSSION

Responsiveness of outcome measures has rarely been investigated in JIA. We previously found that the physician and the parent global were the most responsive measures in a non-controlled trial of methotrexate in patients with polyarticular JIA.²⁴ In another group of JIA patients with oligoarthritis, the physician global and the articular indices proved most responsive, whereas the CHAQ was poor at detecting clinical change.²⁵ In these studies, however, we only relied on the traditional responsiveness statistics and did not examine the discriminative ability of the instruments.

The objective of the present study was to compare the relative responsiveness of the conventional condition specific measures and a generic HRQoL instrument for outcome assessment of IAC injections in 44 patients with oligoarticular JIA. Patients receiving IAC injections are expected, on average, to experience considerable improvement of symptoms in the injected joints.²⁶ Thus, the patients we chose formed a suitable cohort to investigate responsiveness.

In our analysis, the physician global showed the best performances in detecting the baseline versus 6 month change and in discriminating improved from non-improved patients, when either the physician’s or the parent’s external criterion was used. This finding is in keeping with our previous observations and suggests that the physician global is the most powerful endpoint for the assessment of treatment outcome in JIA. The parent global and the parent pain revealed much less ability to detect clinical change and poorer discriminative ability compared with the physician global. Because the baseline value for both parent’s subjective assessments was much lower than that of the physician global, this finding suggests a distinctive difference in the perception of the patient’s status between the physician and the parent, with the parent having a more optimistic view of their children’s disease than the physician. However, because parents and clinicians may have different perspectives on outcomes in arthritis, we cannot establish whether the “objective” physician’s evaluation is more important and realistic than the parent’s “subjective” perception of improvement. Furthermore, the fact that all evaluations were made by the parents instead of the patients themselves could affect their reliability.

The articular indices proved fairly responsive with the traditional responsiveness statistics and demonstrated good discriminative ability, particularly using the physician’s external criterion. The good responsiveness of the articular indices may be due to the fact that we limited the assessment of baseline versus 6 month change to the injected joints, and their good correlation with the physician’s external criterion suggests that the joint examination was the major determinant of the physician’s estimation of the treatment outcome.

The comparison of the condition specific functional ability measure (the CHAQ) with the generic HRQoL questionnaire (the CHQ) showed that the psychosocial summary score of the CHQ was most responsive to change, and that both subscales of the CHQ had superior discriminative ability compared with the CHAQ. Other investigators have also reported that the CHQ is more responsive to change than the CHAQ in JIA, but found that the physical summary score performed better than the psychosocial score.^11,12 Because children who are candidates for IAC injection usually have severe articular symptoms, such as pain, joint contracture, or large effusion, the rapid clinical improvement that often ensues may have a relevant impact on their psychosocial health, leading to a detectable improvement in their emotional wellbeing, behaviour, or self esteem. However, since the IAC injection is a stressful experience for both the patient and the parents, the good performance of the psychosocial component of CHQ observed in our study may partially reflect strong wishful thinking by the parent regarding the treatment outcome. The better results provided by the CHQ compared with the arthritis specific instrument (CHAQ) are in contrast with the expectation that generic instruments, which contain more items that may be less relevant to a particular disease, are less responsive to clinical change. The poor performance of the CHAQ is not surprising, however, because this study was concentrated on children with oligoarticular disease who had very low baseline scores (0.36 (0.49)), which made a considerable change impossible, and constitute the JIA subgroup in which the CHAQ has been shown to have low discriminative value.²⁵ The ESR and the CRP proved poorly responsive and had a low discriminative power, suggesting that these variables had little influence on the physician’s external judgment.

We must acknowledge possible limitations associated with the data presented in this report. Firstly, for any method of calculating responsiveness, the choice of external criteria for change is disputable. We chose both the physician’s and the parent’s own global perception of the treatment outcome to obtain a more comprehensive evaluation of the different disease domains and outcome expectations, although we do not imply that either assessment could be considered as the correct one. Furthermore, as the physician’s and the parent’s assessments, although performed in a different way, were used both as “traditional” outcomes (condition specific measures) and as “gold standard” (external criteria), the fact that one is used as a gold standard for the other may partially represent a kind of “circular proof”. It should also be recognised that our comparisons are based on outcomes of patients who received IAC injections and may not be generalisable to other patient groups or interventions. Moreover, the comparison of the relative performance of the different measures is limited by the fact that we did not assess their validity and did not formally test whether the observed differences were statistically significant.

In conclusion, we found that overall responsiveness statistics and examination of the discriminative ability to distinguish clinically important change consistently ranked the subjective physician’s assessment of the disease activity as the most responsive outcome measure in our patients with JIA. The superior performance of the generic HRQoL assessment tool compared with the arthritis specific instrument CHAQ deserves further investigation.