Article Text
Abstract
Objectives To develop and validate the cut-offs in the Juvenile DermatoMyositis Activity Index (JDMAI) to distinguish the states of inactive disease (ID), low disease activity (LDA), moderate disease activity (MDA) and high disease activity (HDA) in children with juvenile dermatomyositis (JDM).
Methods For cut-off definition, data from 139 patients included in a randomised clinical trial were used. Among the six versions of the JDMAI, JDMA1 (score range 0–40) and JDMAI2 (score range 0–39) were selected. Optimal cut-offs were determined against external criteria by calculating different percentiles of score distribution and through receiver operating characteristic curve analysis. External criteria included the modified Pediatric Rheumatology International Trials Organization (PRINTO) criteria for clinically ID in JDM (for ID) and PRINTO levels of improvement in the clinical trial (for LDA and HDA). MDA cut-offs were set at the score interval between LDA and HDA cut-offs. Cut-off validation was conducted by assessing construct and discriminative ability in two cohorts including a total of 488 JDM patients.
Results The calculated JDMAI1 cut-offs were ≤2.4 for ID, ≤6.6 for LDA, 6.7–11 for MDA and >11 for HDA. The calculated JDMAI2 cut-offs were ≤5.2 for ID, ≤8.5 for LDA, 8.6–11.3 for MDA and >11.3 for HDA. The cut-offs discriminated strongly among disease activity states defined subjectively by caring physicians and parents, parents’ satisfaction or non-satisfaction with illness outcome, levels of pain, fatigue, physical functional impairment and physical well-being.
Conclusions Both JDMAI1 and JDMAI2 cut-offs revealed good metrologic properties in validation analyses and are, therefore, suited for application in clinical practice and research.
- Dermatomyositis
- Outcome Assessment, Health Care
- Autoimmune Diseases
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Recently, the first composite disease activity score for juvenile dermatomyositis (JDM), named Juvenile DermatoMyositis Activity Index (JDMAI), has been developed and validated.
To aid in interpretation of JDMAI scores, criteria (ie, cut-off values) are needed for defining various levels of JDM activity.
WHAT THIS STUDY ADDS
This study defines the cut-off values in the JDMAI that correspond to the states of inactive disease and low, moderate and high disease activity in JDM.
The cut-offs have been validated in a large patient sample.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The JDMAI cut-offs may represent suitable treatment targets and may help improve disease outcome.
Introduction
Juvenile dermatomyositis (JDM) is the most common idiopathic inflammatory myopathy in childhood. It is a systemic autoimmune vasculopathic disease that affects muscle and skin, but may also involve visceral organs, especially the bowel and lung, and is characterised by poorly understood complications, namely dystrophic calcinosis and lipodystrophy.1 2 Despite markedly reduced mortality rates for JDM over the last 50 years, there are still many patients who are treatment resistant and experience chronic disease activity. These patients are at risk of developing disease-related or treatment-related damage and functional disability, which may have a marked impact on their quality of life.3–7
In recent years, the treatment of JDM has been made more rational through the proposal of innovative treatment approaches in uncontrolled studies,8–10 the scrutiny of novel and traditional medications in randomised controlled trials,11 12 and the publication of consensus-based treatment recommendations.13 14 There is nowadays growing interest in assessing new therapies that target various recently discovered pathways implicated in the pathogenesis of idiopathic inflammatory myopathies, including JDM.15
To substantiate these therapeutic advances, there is the need for sensitive, precise and feasible measures of disease activity. A suitable and pragmatic approach to the measurement of disease activity in JDM can be based on the so-called composite disease activity scores (DAS). These tools are designed to quantify the absolute level of disease activity by providing one summary number on a continuous scale. Recently, the first composite DAS for JDM, named Juvenile DermatoMyositis Activity Index (JDMAI), has been developed and validated.16
To aid in the interpretation of the scores obtained with the JDMAI, criteria (ie, cut-off values) are needed for defining various levels of JDM activity. These criteria may provide simple and intuitive reference values for monitoring of disease course over time in an individual patient or for comparing the disease status across single patients or patient groups. Furthermore, they may support decisions about enrolment into clinical trials as well as requirements for changes in therapies and for establishing therapeutic goals in the treat-to-target strategy.
This study was undertaken to determine and validate cut-off values in the JDMAI that correspond to the states of inactive disease (ID), low (or minimal) disease activity (LDA), moderate disease activity (MDA) and high disease activity (HDA). Of note, due to the similarity in the structure between the JDMAI and the Juvenile Arthritis Disease Activity Score (JADAS), a composite DAS for juvenile idiopathic arthritis developed by our group,17 the methods used for the definition and validation of the cut-off values in the JDMAI were similar to those previously employed for the establishment of the JADAS cut-offs.18–21
Methods
Composition and calculation of the JDMAI versions used in the study
The JDMAI combines the following four key measures of disease activity in JDM: (1) physician’s global rating of overall disease activity (PhGA) on a 0–10 Visual Analogue Scale (VAS) (where 0=no activity and 10=maximum activity); (2) parent’s/patient’s global rating of patient’s overall well-being (PaGA) on a 0–10 VAS (where 0=best and 10=worst); (3) measurement of muscle strength and (4) assessment of skin disease activity. In the validation study of the JDMAI,16 six versions of the instrument were tested, which differed in the measures used to assess items 3 and 4. Measurement of muscle strength was made through the hybrid Manual Muscle Testing 8 (MMT)/Childhood Myositis Assessment Scale (CMAS) (hMC) (score range 0=worst to 100=best)22 in JDMAI1 and JDMAI2, the MMT8 (score range 0=worst to 80=best)23 24 in JDMAI3 and JDMAI4, and the CMAS (score range 0=worst to 52=best)25 in JDMAI5 and JDMAI6. To normalise the difference in score range, improve score distribution and avoid giving muscle strength a dominant weight in the index, scores of all muscle tools were expressed in deciles. Furthermore, scores were reversed to give them the same direction (ie, 0=best to 10=worst) as the other JDMAI components.16 To estimate the activity of skin disease, the physician’s global assessment of the activity of skin disease on a 10 cm VAS (skin activity VAS, score range 0=no activity to 10=maximum activity) was included in JDMAI1, JDMAI3 and JDMAI5, and the skin component of the DAS (DAS skin, score range 0=no activity to 9=maximum activity)26 was included in JDMAI2, JDMAI4 and JDMAI6.
Because for the measurement of muscle strength, we favour the use of the hMC, which is more comprehensive than the MMT8 and more feasible than the CMAS,22 we chose for the present study the JDMAI versions that include this tool. However, because there are no universally agreed instruments to quantify skin disease in JDM, we decided not to choose among the two skin assessment scales. For these reasons, we focused our study on JDMA1 and JDMA2, whose score range is 0–40 and 0–39, respectively.16
Patient population used for the development of JDMAI cut-offs
A dataset of 139 patients enrolled in a randomised controlled trial conducted by the Pediatric Rheumatology International Trials Organization (PRINTO) aimed to compare prednisone alone versus prednisone plus methotrexate versus prednisone plus cyclosporine in newly diagnosed patients with JDM 12 was used for the determination of JDMAI cut-off values. The clinical features of patients enrolled in this trial have been reported elsewhere.12
Definitions of disease activity states
The state of ID was defined, according to the modified PRINTO criteria for clinically ID in JDM,27 28 as a PhGA≤0.2 and at least two of the three following criteria: (1) creatine kinase ≤150 U/L, (2) CMAS≥48 and (3) MMT8≥78. The disease state of all patients enrolled in the above-mentioned trial who had achieved at least a JDM PRINTO 70 level of improvement7 at 6 months was defined as LDA. The disease state of all patients enrolled in the PRINTO trial who were non-responders at 6 months, that is, who had not achieved a JDM PRINTO 20 level of improvement,7 was defined as HDA. The state of MDA was defined as a state in between the states of LDA and HDA.
Patient populations used for the validation of JDMAI cut-offs
Two patient samples were used to validate the selected cut-off values. The first sample comprised 213 JDM patients followed in standard clinical care at 13 international paediatric rheumatology centres and evaluated prospectively at baseline and after a median of 5.9 months. The clinical features of these patients have been reported elsewhere.16 In addition to collecting the traditional physician’s centred outcome measures, at the time of the visit the caring physician was asked to rate subjectively the child’s disease state as ID, LDA, MDA or HDA. Furthermore, at every visit the parents of the enrolled patients were asked to make a subjective rating of the child’s disease state as remission, continued activity or flare. To facilitate understanding of disease states by parents and children, remission was defined as ‘complete absence of symptoms’, continued disease activity as ‘continuing presence of symptoms’ and flare as ‘recurrence of symptoms after a period of complete well-being’. The parents were also asked to answer a question about satisfaction with the present symptom state. The question, ‘Considering all the ways the illness affects your child, would you be satisfied if his/her condition remained stable/unchanged for the next few months?’ was to be answered as ‘yes’ or ‘no’.29
The second sample was composed of 275 patients with active JDM enrolled in a multinational study aimed to validate prospectively the provisional PRINTO/American College of Rheumatology/European Alliance of Associations for Rheumatology disease activity core set for the assessment of response to therapy in JDM.7 For the purposes of the study, only the baseline evaluations were retained.
For sake of brevity, the first dataset will hereafter be named as ‘routine sample’ and the second dataset as ‘PRINTO sample’.
Selection of cut-off values
Optimal cut-off values were determined against external criteria (ie, the various disease states, as defined above) by calculating the 10th and 25th percentile (for the ID cut-offs), the 30th and 40th percentile (for the LDA cut-offs), and the 75th and 90th percentile (for the HDA cut-offs) of cumulative score distribution and through receiver operating characteristic (ROC) curve analysis. The choice of the final cut-off values was based on clinical and statistical grounds. In the absence of a specific definition (see above), the cut-offs for the state of MDA were not calculated through statistical analysis but were set at the score interval between the cut-offs for LDA and HDA.18
Validation analyses
Because the JDMAI is primarily proposed for use in clinical practice, validation of the cut-offs was focused on the evaluation of their performance against outcome measures used in routine care.18 Validation procedures were based on the assessment of capacity of the cut-offs to discriminate between: (1) the different disease activity states assessed subjectively by the caring physicians; (2) the different disease activity states assessed subjectively by the parents; (3) parent’s satisfaction or dissatisfaction with the child’s disease status; (4) the degree of pain, as rated by a parent on a 21-numbered circle VAS, where 0=no pain and 10=very severe pain; (5) the level of fatigue, as assessed by a parent on a 21-numbered circle VAS, where 0=no fatigue and 10=very severe fatigue; (6) the absence or presence of physical disability, defined as a Childhood Health Assessment Questionnaire (CHAQ) score of 0 or >0, respectively30; (7) the absence or presence of cumulative damage due to myositis, defined as a Myositis Damage Index extent of damage score, child version, of 0 or >0, respectively31 and (8) a normal or impaired health-related quality of life, defined as a Child Health Questionnaire physical summary score (CHQ-PhS) or psychosocial summary score of ≥40 or <40, respectively.32 33
Quantitative data were compared by means of the Kruskal-Wallis test and percentages through the χ2 test. All statistical tests were two sided, and p values less than 0.05 were considered significant. The statistical packages used were Statistica (release V.9.1, StatSoft) and Stata (release V.11.0, StataCorp).
It was not possible to involve patients or the public in the design and conduct of this research.
Results
Table 1 shows the JDMAI1 and JDMAI2 scores at baseline and at 6-month visit in the aforementioned JDM PRINTO trial.12 As expected, the scores of both tools were quite high at study entry as the trial population was composed of newly diagnosed patients. The scores decreased markedly at 6-month evaluation as a result of treatment interventions.
Selection of the optimal cut-offs for classification of specific disease activity states
The JDMAI1 and JDMAI2 cut-offs obtained with the different statistical approaches are shown in tables 2 and 3. As expected, the cut-offs for ID were the lowest and the values increased progressively for the states of LDA, MDA and HDA. The following criteria were used to select the final cut-offs: specificity was considered more relevant than sensitivity to identify the cut-offs for the states of ID and LDA, in order to reduce the risk of misclassifying patients whose disease was actually active. However, a minimum sensitivity of 75% was requested to ensure adequate face validity of the criteria. Conversely, in selecting the final cut-off values for HDA we gave more importance to sensitivity, that is, to the proportion of patients with active disease who were correctly classified, in order to reduce the risk of misclassifying patients whose disease was active. However, a minimum specificity of 75% was required to minimise the rate of misclassification of patients with LDA/MDA as having HDA. As stated above, the cut-offs for MDA were set at the interval between the cut-offs for LDA and HDA.18
The optimal JDMAI1 and JDMAI2 cut-off values that were selected for the various disease states were those identified by ROC curve analysis and are shown in table 4.
Results of validation analyses
In the routine sample, the percentage of visits in which patients were judged subjectively by the caring physician as being in the state of ID or LDA was greater among patients with JDMAI1 or JDMAI2 scores below the cut-off values for ID or LDA, whereas the percentage of visits in which patients were judged by the caring physician as being in the state of MDA or HDA was greater among patients with a JDMAI1 or JDMAI2 score within the interval corresponding to MDA or above the HDA cut-off (see figure 1 for JDMAI1 and online supplemental figure S1 for JDMAI2). Likewise, the proportion of visits in which the JDMAI1 and JDMAI2 scores were below the cut-off value for ID was higher among patients judged subjectively by their parents as being in the state of ID than as having continued activity or flare (online supplemental figure S2 for JDMAI1, data not shown for JDMAI2). In the same sample, the percentage of visits in which parents were satisfied with their child’s disease state was greater among patients with a JDMAI1 or JDMAI2 scores below the cut-off values for ID or LDA, whereas the percentage of visits in which the parents were not satisfied by their child’s disease state was greater among patients with a JDMAI1 or JDMAI2 within the interval corresponding to MDA or above the HDA cut-off (see figure 2 for JDMAI1 and online supplemental figure S3 for JDMAI2). Notably, all patients included in the JDM PRINTO trial had a baseline score above the HDA cut-off for both JDMAI1 and JDMAI2.
Supplemental material
The level of pain was lowest in patients with JDMAI1 or JDMAI2 scores below the cut-off value for ID, and proportionally greater in patients with a JDMAI1 or JDMAI2 scores indicating higher disease activity states, both in routine (see online supplemental figure S4 for JDMAI2, data not shown for JDMAI1) and PRINTO (data not shown) datasets. Similar results were observed in the routine sample in relation to the degree of fatigue, which was more pronounced in patients categorised by the JDMAI1 or JDMAI2 score as being in a state of MDA or HDA (see figure 3 for JDMAI1 and online supplemental figure S5 for JDMAI2), as expected.
In the PRINTO dataset, the percentage of patients who had normal physical function (ie, a CHAQ score=0) was greater among those who had a JDMAI1 or JDMAI2 below the cut-offs for ID or LDA, whereas the percentage of patients who had a CHAQ score >0 was greater among those who had a JDMAI1 or JDMAI2 in the interval corresponding to MDA or above the HDA cut-off (See figure 4 for JDMAI1 and online supplemental figure S6 for JDMAI2). A proportionally greater impairment of HRQL in the physical domain (CHQ-PhS), but not in the psychosocial domain (PsS), was seen from patients with the JDMAI1 or JDMAI2 below the cut-offs for ID/LDA to patients meeting the criteria for higher disease activity states (see online supplemental figures S7 and S8 for JDMAI1, data not shown for JDMAI2). The amount of cumulative damage did not differ among patients meeting the various JDMAI1 or JDMAI2 disease activity states (see online supplemental figure S9, results shown only for JDMAI2).
Discussion
In this study, we sought to determine the cut-offs in the JDMAI1 and JDMAI2 that correspond to the states of ID, LDA, MDA and HDA in JDM. Cut-offs definition was performed using a multinational dataset of 139 patients enrolled in a multinational multicentre clinical trial. The selected cut-offs were cross-validated in two independent multinational cohorts comprising 213 JDM patients followed longitudinally in routine clinical care and 275 patients with active JDM enrolled in a study aimed to devise a disease activity core set. The size of the patient samples, which is large for a rare disease such as JDM, and the wide geographical distribution of the centres make the study findings likely generalisable to patients with various JDM phenotypes and treated with different approaches.
For the definition of the cut-offs, we applied a methodology similar to that previously used for the establishment of the JADAS cut-offs for disease activity states in juvenile idiopathic arthritis.18–20 The selected cut-off values were those yielded by ROC curve analysis, which exhibited the best balance between sensitivity and specificity. The good performance of the cut-offs is corroborated by their sensitivity and specificity consistently above or close to 80% and by the AUCs above or close to 0.90.
In cross-validation analyses, the cut-offs showed strong ability to discriminate between different disease activity states based on the subjective perception of paediatric rheumatologists or parents from different regions of the world. The cut-offs for ID and LDA were met more frequently by patients whose disease state was judged by the caring physician or the parent as remission, or was deemed acceptable by the parent. Conversely, the cut-offs for HDA were met more commonly by patients judged by the caring physician or the parent as having continued disease activity or disease flare, or deemed by the parent as not being in an acceptable symptom state. The cut-offs proved also able to discriminate between the levels of pain and fatigue, which were lowest in patients who met the ID cut-off and proportionally greater in patients who were in LDA, MDA and HDA by the cut-offs.
The observation that patients meeting the cut-offs for ID and LDA had lesser physical disability and better HRQL in the physical domain, whereas those whose JDMAI scores were above the thresholds for MDA and HDA had greater impairment of these two health domains indicate that the cut-offs possess good construct validity and may potentially predict prognosis. The lack of difference in the proportion of patients who met the various cut-offs in relation to the level of HRQL in the psychosocial domain was expected, as this aspect of quality of life is influenced by many factors external to disease activity.32 33 The comparability of the degree of cumulative damage across patients meeting the different cut-offs reinforces the role of the JDMAI as a specific measure of disease activity that is not affected by the different construct of disease damage.
Among the two JDMAI versions tested, we favour the use of the JDMAI1 because it revealed overall better performances in validation analyses and included a simpler measure of skin disease activity, which makes it more feasible for regular use in daily practice.
Our results should be interpreted in light of some potential caveats. The reference criteria adopted for the definition of disease activity states against which to determine the cut-offs, namely the modified PRINTO criteria for clinically ID and the PRINTO level of clinical improvement,7 27 28 34 35 comprise some JDMAI components, which may raise issues related to circular assessment. However, the strong ability of the cut-offs to discriminate between disease activity states defined subjectively by paediatric rheumatologists and parents demonstrates that their value corresponds well with the perception of disease activity level of the caring stakeholders. We arbitrarily set the cut-off for MDA in between the cut-offs for LDA and HDA. A consensus definition or a judgmental approach (ie, explicitly asking physicians and/or parents their opinion on what they would consider MDA) might have led to cut-off values with higher face validity and relevance in practice. The hMC, which was chosen to measure muscle strength, may not be familial to many paediatric rheumatologists. However, it can be easily calculated by summing the score of the MMT8 to that of 3 of the 14 items of the CMAS, with only a slight modification in the score of the floor rise item.22 Note that because the muscle strength component of the JDMAI is measured in deciles and we previously found a close correlation between the hMC and both MMT8 and CMAS,17 either of the latter instruments can be used interchangeably with the hMC. For the measurement of skin disease, we used the skin VAS—derived from the Myositis Disease Activity Assessment VAS that is part of the Myositis Disease Activity Assessment Tool31—for JDMAI1, and the DAS skin for JDMAI2. However, there is no universal agreement about which tool is best suited to assess skin disease in JDM,36 and it is anticipated that the JDMAI might need to be revised when new well-designed and validated skin-specific instruments for JDM become available.31 We could not account for the increasingly recognised heterogeneity of JDM in terms of histopathological findings on muscle biopsy samples and myositis-specific autoantibody profile.37–40 We should finally acknowledge that the JDMAI assesses specifically the two major systems affected in JDM (skeletal muscles and skin), but neglects other potentially, though less commonly, involved organs/systems, such as the gastrointestinal, pulmonary and cardiac. Involvement of these organs is of foremost clinical importance and can be overlooked, especially in patients with minor muscle or skin involvement. Lung involvement is often not routinely well analysed or evaluated incompletely without assessment of DLCO. Thus, the JDMAI cut-offs can only be used in patients without gastrointestinal, heart or lung involvement.
In summary, we have developed the criteria for the definition of disease activity states in JDM based on the JDMAI1 and JDMAI2. In validation analyses, the cut-offs revealed strong ability to discriminate between disease activity states defined subjectively by caring physicians and parents as well as between different levels of pain, fatigue, physical functional disability and physical well-being. The cut-offs represent an additional clinical tool that, if applied regularly in daily practice, may allow tighter therapeutic control of disease, support the optimisation of treatment on an individual patient basis and help prevent the development of disease damage and physical disability.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Comitato di Etica per la Ricerca Scientifica e Biomedica e per la Sperimentazione Clinica (Reference number: 41/2013). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
This work was partially supported by the Fundación Española de Reumatología (FER) and by the UK Juvenile Dermatomyositis Research Group (JDRG). The results of this work have been previously presented at the Global Conference on Myositis 2022 (abstract in press), the 28th European Pediatric Rheumatology Congress (PReS 2022) (Pediatric Rheumatology 2022,20(Suppl2):P276), and the congress of the Società Italiana di Reumatologia Pediatrica (Reumaped 2022).
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Study conception and design: AR, SR, AC and NR. Acquisition of data: CB, PD, AGB-M, TH, MT, VV, CW and LC. Analysis and interpretation of data: AP and AR-G. Guarantor: SR.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.