Article Text
Abstract
Objective Systemic sclerosis Impact of Disease (ScleroID) is the first comprehensive patient-reported outcome measure (PROM) specifically developed for systemic sclerosis (SSc). We investigated the performance of ScleroID in patients with diffuse cutaneous SSc (dcSSc), as a prerequisite for its use in randomised controlled trials (RCTs) testing potentially disease-modifying drugs.
Methods All patients with dcSSc from the large, multicentric, ScleroID cohort were included. SSc-Health Assessment Questionnaire (HAQ), EuroQol-5 Dimensions and 36-item Short Form Health Survey (SF-36) were used as comparators. The study includes a longitudinal arm with a reliability visit at 7±3 days and a 12 months follow-up visit. The performance of ScleroID in dcSSc was assessed according to the Outcome Measures in Rheumatology filter.
Results In total, 152 dcSSc patients were analysed (29% male, median age 54 years). ScleroID reflected well the disease impact of dcSSc, showing a good construct validity with high Spearman’s correlation coefficients with comparators (SSc-HAQ, 0.79, 95% CI (0.69, 0.86); HAQ-Disability Index, 0.72 95% CI (0.60, 0.80); SF-36 physical score, −0.69 95% CI (−0.77, –0.60)). The internal consistency was strong (Cronbach’s alpha 0.87, split-half reliability coefficient 0.88).
In the longitudinal arm, 44 patients had a reliability visit and 113 had a follow-up visit, of whom 19/113 (17%) reported a significant change (11 improved, 8 worsened). ScleroID showed a good consistency and discriminative ability with excellent test–retest reliability (intraclass correlation coefficient 0.89, 95% CI (0.84, 0.92)) and moderate sensitivity to change (standardised response mean −0.63 in the improved subgroup and 0.48 in the worsened subgroup), but superior to the comparators.
Conclusion The European Alliance of Associations for Rheumatology (EULAR) ScleroID performs well for patients with dcSSc. This supports its inclusion and regular assessment as PROM in RCTs.
- Patient Reported Outcome Measures
- Scleroderma, Systemic
- Connective Tissue Diseases
- Health-Related Quality Of Life
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- Patient Reported Outcome Measures
- Scleroderma, Systemic
- Connective Tissue Diseases
- Health-Related Quality Of Life
WHAT IS ALREADY KNOWN ON THIS TOPIC
Systemic sclerosis Impact of Disease (ScleroID) has been successfully validated as a patient-reported outcome measure (PROM) for systemic sclerosis (SSc) in an unselected cohort of patients.
WHAT THIS STUDY ADDS
In this post hoc analysis of the data from the original ScleroID study, we could show its superior performance to comparators (SSc-Health Assessment Questionnaire, EuroQol-5 Dimensions, 36-item Short Form Health Survey) in diffuse cutaneous SSc (dCSSc) patients.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This analysis provides evidence for further use of ScleroID as a disease-specific PROM in clinical trials and routine care for patients with dcSSc.
Introduction
Diffuse cutaneous systemic sclerosis (dcSSc) is the most severe form of SSc, often associated with multiple organ involvement and significant morbidity and mortality. There is a pressing, unmet need for efficient therapies, which drives the development and testing of novel therapeutic agents in randomised controlled trials (RCTs) in SSc.1 Due to their increased morbidity and mortality, dcSSc patients are most frequently recruited into clinical trials.
Patient-reported outcome measures (PROMs) play an important part of patient-centred medicine and hence are key outcome measures for RCTs.2 3 In RCTs, PROMs allow assessment of health-related quality of life, treatment effect and safety as perceived by patients. PROMs are mandatory for all RCTs and are highly relevant for regulatory authorities when evaluating applications for marketing authorisation of new therapeutic agents.1 2 4 5
However, selecting the ideal PROM to reflect the outcome of interest in SSc-RCTs is often difficult, on one side due to the paucity of specific and validated PROMs, on the other side due to the heterogeneity of the disease, both in terms of clinical manifestations and disease progression.1 6 Most SSc-RCTs relied on a set of generic, legacy PROMs that showed good performance in SSc, such as the Health Assessment Questionnaire Disability Index (HAQ-DI) and its adaptation, the Scleroderma HAQ, the medical outcomes study 36-item Short Form Health Survey (SF-36)7 and the EuroQol-5 Dimensions (EQ-5D).8
Most of the available and validated SSc-specific PROMs focus on particular aspects of the disease. Examples include instruments such as the Raynaud Condition Score, the Scleroderma Skin Patient-Reported Outcome or the new Hand Disability in Systemic Sclerosis-Digital Ulcers tool.1 9 10
To date, none of the RCTs in SSc meeting their primary endpoint in interstitial lung disease have shown a meaningful change in PROMs when they were used as secondary outcome measures for treatment effect.1 This might be attributed to limitations of the investigational drug itself but also due to the insufficient sensitivity to change of the PROMs available.1 This highlights the need for developing better PROMs, which can reflect how patients feel, cope with and respond to the use of an investigational product in clinical trials.11 Considering the predominant focus of RCTs on patients with dcSSc, a validated PROM to reflect the disease burden experienced by patients with dcSSc is crucial.
Addressing this unmet need, we have recently developed and validated the Systemic sclerosis Impact of Disease (ScleroID) questionnaire, which has received the endorsement of the European Alliance of Associations for Rheumatology (EULAR). This is the first comprehensive PROM specifically developed by SSc patients and experts to reflect the global disease impact of SSc, which showed a good performance in a large clinical validation study.12 The development of ScleroID was specifically designed for SSc, based with adaptations on the previous, successful development of similar EULAR composite measures for rheumatoid arthritis and psoriatic arthritis.13–15
In the current study, we report a novel, detailed analysis of the performance of ScleroID in the subset of patients with dcSSc, adhering to the guidelines established by Outcome Measures in Rheumatology (OMERACT), thus adding important information for its future use in clinical trials.16
Patients and methods
The ScleroID development and validation study was a multicentric collaboration including 11 European expert SSc centres and patient research partners.12 Briefly, the study included the cross-sectional analysis of a large baseline cohort consisting of 472 SSc patients, along with a longitudinal component featuring a reliability visit at 7±3 days (109 patients) and a 12-month follow-up visit (113 patients). At all visits, patients completed the ScleroID questionnaire as well as the SSc-HAQ, EQ-5D and SF-36.12
In the current study, we focus on analysing all patients diagnosed with dcSSc from the original ScleroID cohort.
The performance of ScleroID in dcSSc was assessed according to the OMERACT guidelines, including the major pillars of truth, discrimination and feasibility.16 17 Truth encompasses face and content validity (‘Is the measure applicable and does it make sense?’, floor/ceiling effect), construct validity (‘Does the PROM measure what it should measure?’) and internal consistency (‘Do the PROM items cover all the aspects they are supposed to?’). Spearman correlations between ScleroID and the other established PROMs (SSc-HAQ, EQ-5D, SF-36), as well as Cronbach’s alpha and the split-half reliability coefficient, were calculated accordingly.
Discrimination was assessed through test–retest reliability (‘Are the results reproducible in a stable population?’) by calculating the intraclass correlation coefficient (with the reference values below 0.50: poor, between 0.50 and 0.75: moderate, between 0.75 and 0.90: good, above 0.90: excellent18). Patients self-reported their perceived disease status at follow-up by answering a dedicated Likert-scale question (online supplemental methods). Sensitivity to change was consequently assessed to determine whether the tool could distinguish between different groups at follow-up, such as stable versus improved or worsened, using the standardised response mean (SRM).
Supplemental material
Results
Out of 152 dcSSc patients with baseline data analysed, 44 (29%) were male, with a median age of 54 years and a median disease duration of 7 years since the first non-Raynaud symptom. The self-reported disease status was good or very good in a third of patients, acceptable in almost half of the cohort and bad or very bad in 17% of patients (table 1). A detailed cohort description is shown in table 1.
Truth and feasibility
The ScleroID questionnaire was filled in completely by the great majority of dcSSc patients, with minimal percentages of missing data observed among the individual items (online supplemental figure S1). There was no relevant ceiling or floor effect (online supplemental figure S2) and the individual scores of the ScleroID items were distributed uniformly across the cohort (online supplemental figure S3).
ScleroID showed a good construct validity with high Spearman’s correlation coefficients with the other PROMs (SSc-HAQ, 0.79, 95% CI (0.69, 0.86); HAQ-DI, 0.72 95% CI (0.60, 0.80); SF-36 physical score, −0.69 95% CI (−0.77, –0.60)) (online supplemental table S1 and table 2).
Furthermore, the internal consistency was strong, according to Cronbach’s alpha coefficient of 0.87 and to the split-half reliability coefficient of 0.88 (table 2).
Discrimination
In the longitudinal follow-up, 44 patients with dcSSc underwent a test–retest reliability visit at 7±3 days. ScleroID showed excellent consistency, with a calculated intraclass correlation coefficient of 0.89, 95% CI (0.84, 0.92).
In total, 113 patients had a 12-month follow-up visit, of whom 19/113 (17%) reported a meaningful change (11 improved, 8 worsened). ScleroID showed a moderate sensitivity to change, nonetheless superior to the other PROMs, as shown in table 2 and, in more detail, in table 3. Considering the sensitivity to change separately, depending on the direction of change, ScleroID showed a global SRM of −0.63 in the patients who improved, respectively, an SRM of 0.48 in those who worsened (and an SRM of −0.12 for patients with no change; table 3). The superiority to the other PROMs in terms of SRM was evident in both improved and worsened subgroups (table 3). The ScleroID values changed overall from baseline median 3.22 (IQR 1.7–4.7) to 3.27 (IQR 2.27–4.77) (patients who improved: 2.73 (IQR 1.74–3.88) to 1.78 (IQR 0.56 to 2.53) and patients who worsened: 4.77 (IQR 3.09–5.34) to 5.48 (IQR 4.71–5.90).
Discussion
This in-depth analysis, specifically focusing on dcSSc patients, shows overall very good performance of ScleroID in this subgroup of individuals with more severe disease manifestations. The questionnaire is feasible to apply in a clinical setting and has the ability to reflect/capture the impact of dcSSc from the patients’ own perspective across all relevant disease-related health dimensions. The good correlation with other established/validated PROMs and the strong internal consistency provide evidence of this.
Furthermore, the ScleroID questionnaire showed a good discriminative ability in the longitudinal analysis. First, it showed very good reproducibility in stable patients at repeated measurements. In addition, the sensitivity to change at 12 months follow-up was higher than for the other PROMs, with a moderate SRM, consistent in the improving/worsening subgroups. However, the low number of patients with a self-reported change at follow-up impairs a reliable evaluation of the sensitivity to change of the individual items of ScleroID based solely on these data. This limitation remains for exploration in future studies, particularly in the setting of a clinical trial, where besides the global disease impact, specific effects on certain health domains might be of particular interest.
Overall, the good performance of ScleroID in this cohort of patients with dcSSc further supports the findings from the unselected, original SSc validation cohort and from a large, unselected, monocentric cohort.12 19
Our study is the first to analyse the performance of ScleroID in patients with dcSSc. Strengths of our study further include the multicentre, large cohort of patients with SSc, the thorough design and methodology following the OMERACT filter and the inclusion of several, widely used PROMs as comparators (SSc-HAQ, EQ-5D and SF-36). The data presented in this study support the decision of including ScleroID as a PROM in clinical trials focusing on patients with dcSSc and ist evaluation in clinical practice.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by ethics committee of the canton of Zurich, EK-839 and BASEC-Nr. PB_2016-01515 and BASEC-Nr. 2018-02165. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We sincerely thank the patients’ representatives who contributed to the ScleroID study, without whom this project would not have been possible: Ann Kennedy, Kim Fligelstone, Heleen Lever, Yanne Perriault, Anna Vegh, Susanne Tuppeck, Silvia-Daniela Sandulescu, Mervat Gaafar, Barbara Zappitello, Rachida Amrouch, Grazia Tassini, Helene Kambourakis, Elisabeth Scheel, Stephan Houbertz, Beata Garay Toth, Ana Marcela Badea, Stefana Dumitru, Alicia García Oliva, Begoña María Gorricho Corta, Johanna Berglind, Natalie Perruchoud, Alice Martins Correia, Richard Dodds, Nicola Whitehill. Our thoughts go to Mr. Richard Dodds, who is, sadly, no longer with us.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
X @cosimobruni
Presented at We would like to acknowledge that a preliminary analysis of this work was presented at the EULAR 2023 European Congress of Rheumatology: Dobrota R, Garaiman A, Fligelstone K, et al. POS0342 Performance of the EULAR Systemic Sclerosis Impact of Disease (SCLEROID) questionnaire as a patient-reported outcome measure for patients with diffuse systemic sclerosis. Annals of the Rheumatic Diseases 2023;82:418-419.
Contributors MB is the guarantor and accepts full responsibility for the work and the conduct of the study, has access to the data and controls the decision to publish.The authors as listed on the title page of the manuscript have all made substantial contributions which qualifies them as authors. All authors contributed to critical revisions and approved the final version of the manuscript. RD: design of the study, acquisition of data, analysis, interpretation of data, drafting and revising the article. AG: analysis, interpretation of data, drafting and revising the article. KF, ATK: design of the study, interpretation of data, revising the article. AR: interpretation of data, revising the article. YA, PEC, LC, CD, RH, GS, OK-B, CB, MM-C and CM: design of the study, acquisition of data, interpretation of data, drafting and revising the article. UM-L and TK: design of the study, interpretation of data, drafting and revising the article. TH, OD and MB: design of the study, acquisition of data, analysis, interpretation of data, drafting and revising the article.
Funding The original development and validation of the ScleroID was in part supported by a grant from theThe original development and validation of the ScleroID was in part supported by a grant from the
Competing interests RD: speakers bureau for Actelion, advisory board for Boehringer-Ingelheim, grants/research support from Pfizer, Actelion, Iten-Kohaut foundation, Walter und Gertrud Siegenthaler Fellowship, congress participation support from Amgen, Otsuka. AG, KF, ATK, AR, YA, PEC and LC: none reported. CD: consultancy fees from: Janssen, GlaxoSmithKline, Bayer, Sanofi, Boehringer Ingelheim, Roche, CSL Behring, Corbus, Acceleron, Horizon, Arxx, Lilly, Novartis, Certa and research grants to institution from: Abbvie, Servier, Horizon, GlaxoSmithKline. RH, GS, OK-B: none reported.CB: consulting for Boehringer Ingelheim. Research grants from Foundation for Research in Rheumatology (FOREUM), Gruppo Italiano Lotta alla Sclerodermia (GILS), European Scleroderma Trials and Research Group (EUSTAR), Scleroderma Clinical Trials Consortium (SCTC), Scleroderma Research Foundation (SRF), Novartis Foundation for biological-medical research, EMDO Foundation. Educational grants from AbbVie and Wellcome Trust. Congress participation support from Boehringer Ingelheim and Müller-Hartmann foundation. MM-C: none reported. CM received speaker and/or consultancy fees from Janssen-Cilag AG, Boehringer Ingelheim, Deutsche Rheumaakademie, MED Talks Switzerland, Mepha, Novartis and PlayToKnow AG, and travel support for congress from Boehringer Ingelheim. AMG: consulting and congress participation support from Boehringer Ingelheim. Research grant from Foundation for Research in Rheumatology (FOREUM). UM-L, TK and TH: none reported. OD: has/had consultancy relationship with and/or has received research funding from and/or has served as a speaker for the following companies in the area of potential treatments for systemic sclerosis and its complications in the last three calendar years: 4P-Pharma, Abbvie, Acceleron, Alcimed, Altavant, Amgen, AnaMar, Argenx, Arxx, AstraZeneca, Blade, Bayer, Boehringer Ingelheim, Cantargia AB, Corbus, CSL Behring, Galderma, Galapagos, Glenmark, Gossamer, Horizon, Janssen, Kymera, Lupin, Medscape, Merck, Miltenyi Biotec, Mitsubishi Tanabe, Nkarta Inc., Novartis, Orion, Prometheus, Redxpharma, Roivant, EMD Serono, Topadur and UCB. Patent issued 'mir-29 for the treatment of systemic sclerosis' (US8247389, EP2331143). Cofounder of CITUS AG. OD is a member of the editorial board of RMD open. MB has consultancy relationship with and/or received research funding from and/or served as speaker for the following companies in the area of potential treatments for systemic sclerosis and its complications in the last three calendar years: GSK, Amgen, Novartis and outside of this research area: Vifor.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.