Article Text
PDF
Abstract
Objective To develop a multivariable model for predicting the progression of systemic sclerosis-associated interstitial lung disease (SSc-ILD) over 52 weeks.
Methods We used logistic regression models to analyse associations between candidate predictors assessed at baseline and progression of SSc-ILD (absolute decline in forced vital capacity (FVC) % predicted >5% or death) over 52 weeks in the placebo group of the SENSCIS trial. Analyses were performed in the overall placebo group and in a subgroup with early and/or inflammatory SSc and/or severe skin fibrosis (<18 months since first non-Raynaud symptom, elevated inflammatory markers, and/or modified Rodnan skin score (mRSS) >18) at baseline. Model performance was assessed using the area under the receiver operating characteristic curve (AUC).
Results In the overall placebo group (n=288), the performance of the final multivariable model for predicting SSc-ILD progression was moderate (apparent AUC: 0.63). A stronger model, with an apparent AUC of 0.75, was developed in the subgroup with early and/or inflammatory SSc and/or severe skin fibrosis at baseline (n=155). This model included diffusing capacity of the lung for carbon monoxide (DLco) % predicted, time since first non-Raynaud symptom, mRSS, anti-topoisomerase I antibody status and mycophenolate use.
Conclusion Prediction of the progression of SSc-ILD may require different approaches in distinct subgroups of patients. Among patients with SSc-ILD and early and/or inflammatory SSc and/or severe skin fibrosis, a nomogram based on a multivariable model may be of value for identifying patients at risk of short-term progression.
- pulmonary fibrosis
- scleroderma, systemic
- risk factors
Data availability statement
Data are available upon reasonable request. To ensure independent interpretation of clinical study results and enable authors to fulfil their role and obligations under the ICMJE criteria, Boehringer Ingelheim grants all authors access to relevant clinical study data. In adherence with the Boehringer Ingelheim Policy on Transparency and Publication of Clinical Study Data, scientific and medical researchers can request access to clinical study data, typically one year after the approval has been granted by major regulatory authorities or after termination of the development programme. Researchers should use https://vivli.org/ to request access to study data and visit https://www.mystudywindow.com/msw/datasharing for further information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Risk factors for the progression of systemic sclerosis-associated interstitial lung disease (SSc-ILD) have been identified but are not consistent across studies. The course of SSc-ILD for an individual patient remains largely unpredictable.
WHAT THIS STUDY ADDS
Among patients in the SENSCIS trial who had early and/or inflammatory SSc and severe skin fibrosis, the predictors of SSc-ILD progression over 52 weeks in a multivariable model were lower DLco % predicted, earlier SSc, worse skin fibrosis, ATA positivity and not using mycophenolate.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
A nomogram based on the multivariable model developed in this study may be of value in identifying patients with SSc-ILD who are at risk of short-term progression.
Introduction
Systemic sclerosis (SSc) is a complex autoimmune disease characterised by progressive fibrosis of the skin and internal organs.1 Interstitial lung disease (ILD) is a common manifestation of SSc that often results in pulmonary fibrosis and is associated with poor outcomes.2 3 Observational studies in patients with SSc-ILD have identified various patient characteristics that are prognostic of a greater rate of decline in forced vital capacity (FVC) and/or a greater risk of mortality in certain patient populations.2–18 In addition, multivariable prediction models have been developed to predict a decline in lung function,9 mortality19 or progressive pulmonary fibrosis12 in patients with SSc-ILD. However, the risk factors identified for the progression of SSc-ILD are not consistent across studies, and the course of SSc-ILD for an individual patient remains largely unpredictable.
The SENSCIS trial enrolled patients with SSc and at least 10% of their lungs were affected by fibrosis.20 In the placebo group, over 52 weeks, the rate of decline in FVC was 93.3 mL/year,20 and 28.4% of the patients had an absolute decline in FVC % predicted of >5%.21 We used data from the placebo group of the SENSCIS trial to develop multivariable models for prediction of progression of SSc-ILD over 52 weeks.
Methods
Study design
The design of the SENSCIS trial has been described, and the protocol is publicly available.20 Briefly, patients had SSc with their first non-Raynaud symptom in the prior ≤7 years, an extent of fibrotic ILD ≥10% on high-resolution computed tomography (HRCT), FVC ≥40% predicted and diffusion capacity of the lung for carbon monoxide (DLco) 30–89% predicted. Patients taking prednisone ≤10 mg/day and/or stable therapy with mycophenolate or methotrexate for ≥6 months were allowed to participate. Patients were randomised to receive nintedanib or a placebo.
Analyses
These analyses were conducted in patients who received ≥1 dose of placebo. The following baseline characteristics were considered as candidate predictors of SSc-ILD progression: age2 7 8 16 19 22; time since first non-Raynaud’s symptom23 24; high-sensitivity C reactive protein (CRP)25–27; modified Rodnan skin score (mRSS)2 3 8 10; extent of fibrotic ILD on HRCT2 4 16; FVC % predicted2 7 15 16 28; DLco % predicted7 18 22; sex2 3 11 14 15; cutaneous subtype (diffuse cutaneous SSc vs limited cutaneous SSc)7 28; anti-topoisomerase I antibody (ATA) status (positive vs negative)7 15 17 28; honeycombing on HRCT (yes vs no)4 29; ground glass opacities on HRCT (yes vs no)30; mycophenolate use (yes vs no)31; history of gastro-oesophageal reflux disease (yes vs no).12 13 Pairwise correlations between candidate predictors were assessed using the Pearson correlation coefficient.
The prediction models were developed and validated in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.32 We used univariable and multivariable logistic regression models to analyse associations between the candidate predictors and progression of SSc-ILD (defined as an absolute decline in FVC % predicted >5% at week 52 or death up to week 52). Missing data on the candidate predictors were not imputed. Missing FVC values at week 52 were imputed using the worst observation carried forward approach. In multivariable models, variables were selected using stepwise selection with a threshold to enter and stay of p=0.1. Mycophenolate use was treated as a fixed (non-removable) variable. Only main effects, not interaction effects, were considered in the selection process due to the limited sample size.
Analyses were performed in all patients in the placebo group and in a subgroup of those patients with early and/or inflammatory SSc and/or severe skin fibrosis (<18 months since first non-Raynaud symptom, and/or elevated inflammatory markers (CRP ≥6 mg/L and/or platelets ≥330 × 109 /L) and/or mRSS>18) at baseline, which had previously been shown to be associated with a higher risk of SSc-ILD progression in the SENSCIS trial.24
The performance of the models, that is, their ability to discriminate progressors from non-progressors, was assessed using the area under the receiver operating characteristic curve (AUC). The final multivariable model developed in the patients with early and/or inflammatory SSc and/or severe skin fibrosis was also assessed using the Brier score, which ranges from 0 (perfect model) to 0.25 (non-informative model)32 and a calibration plot. To construct the calibration plot, patients were sorted by predicted risk of progression and split into five equal-sized groups. For each group, the average predicted progression risk was plotted against the observed progression risk, that is, the fraction of patients within the group who progressed. The apparent calibration line was derived via regression of the observed risk of progression against the average predicted risk of progression. Internal validation was performed using a bootstrapping approach to correct for over-optimism in the apparent performance measures of the model on the data used for model development. The entire modelling process based on stepwise variable selection was repeated in 1000 bootstrap samples drawn with replacement from the development data. Over-optimism was assessed as the average difference between the bootstrap models’ performance measures (ie, AUC and Brier score) on the bootstrap samples and their performance measures based on the development data.
The final multivariable model developed in the patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline was used to construct a nomogram in which points are assigned to each prognostic factor and then summed to derive a progression score that is mapped to the risk of progression. A plot of the progression score against the predicted risk of progression derived from the model was constructed to assess their relationship. The distributions of the progression score in progressors and non-progressors were plotted.
Results
The baseline characteristics that were considered as candidate predictors for the prediction model are shown in online supplemental table 1 (overall placebo group) and online supplemental table 2 (patients in the placebo group who had risk factors for rapid FVC decline at baseline). These variables did not show high pairwise correlations (online supplemental tables 3 and 4).
Supplemental material
In the overall placebo group (n=288), 86 patients (29.9%) had progression of SSc-ILD over 52 weeks. In univariable models, DLco % predicted was the only characteristic that showed a strong association with the outcome (figure 1). The performance of the final multivariable model for predicting SSc-ILD progression was moderate (apparent AUC of 0.63) (figure 1).
Associations of baseline characteristics with an absolute decline in FVC >5% predicted or death over 52 weeks in the placebo group of the SENSCIS trial. Baseline variables were treated as continuous terms unless indicated otherwise. *OR shown per 1-unit increase for continuous variables. ATA, anti-topoisomerase I antibody; AUC, area under the receiver operating characteristic curve; CRP, C reactive protein; DLco, diffusion capacity of the lung for carbon monoxide; FVC, forced vital capacity; GERD, gastro-oesophageal reflux disease; HRCT, high-resolution computed tomography; ILD, interstitial lung disease; mRSS, modified Rodnan skin score; SSc, systemic sclerosis.
In the subgroup of patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline (n=155), several factors showed stronger associations with the outcome than were observed in the overall placebo group (figure 2). In univariable models, DLco % predicted showed the strongest association with the outcome. The final multivariable model had an apparent AUC of 0.75 (figure 2; online supplemental figure 1A) and an apparent Brier score of 0.17 (indicating moderate accuracy). This model included DLco % predicted, time since the first non-Raynaud symptom, mRSS, ATA status and mycophenolate use (figure 2). The model formula for deriving the predicted progression risk from the predictor values is shown in the online supplemental material. The calibration line of this model had an intercept of −0.04 and a slope of 1.16 (online supplemental figure 1B). The optimism-corrected apparent performance measures derived through internal validation were 0.66 for AUC and 0.20 for the Brier score.
Associations of baseline characteristics with an absolute decline in FVC >5% predicted or death over 52 weeks in patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline in the placebo group of the SENSCIS trial. Baseline variables were treated as continuous terms unless indicated otherwise. *OR shown per 1-unit increase for continuous variables. ATA, anti-topoisomerase I antibody; AUC, area under the receiver operating characteristic curve; CRP, C reactive protein; DLco, diffusion capacity of the lung for carbon monoxide; FVC, forced vital capacity; GERD, gastro-oesophageal reflux disease; HRCT, high-resolution computed tomography; ILD, interstitial lung disease; mRSS, modified Rodnan skin score; SSc, systemic sclerosis.
The nomogram constructed using the final multivariable model in patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline is shown in figure 3. An example patient who has a CRP of 6.1 mg/L, is ATA positive, has a DLco of 60% predicted, has an mRSS of 15, had their first non-Raynaud symptom 2 years ago and is taking mycophenolate has a progression score of 65, which corresponds to a 24% predicted risk of SSc-ILD progression over 52 weeks. The formulae for deriving the progression score from the predictor values, and the predicted progression risk from the progression score, are shown in the online supplemental material. The relationship between the progression score and the predicted risk of SSc-ILD progression derived from the multivariable model is shown in online supplemental figure 2. The distribution of the progression score in progressors and non-progressors is shown in online supplemental figure 3.
Nomogram for prediction of risk of absolute decline in FVC % predicted >5% or death over 52 weeks in patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline. The calculation for an example patient who has a C reactive protein of 6.1 mg/L, is ATA positive, has a DLco of 60% predicted, has an mRSS of 15, had their first non-Raynaud symptom 2 years ago and is taking mycophenolate is shown in red. ATA, anti-topoisomerase I antibody; DLco, diffusion capacity of the lung for carbon monoxide; FVC, forced vital capacity; mRSS, modified Rodnan skin score; SSc, systemic sclerosis.
Discussion
Observational studies have identified various factors associated with the progression of SSc-ILD in specific populations of patients. However, when applied in clinical practice, the individual course of SSc-ILD remains difficult to predict. In our analysis using data from the SENSCIS trial, a strong model to predict the progression of SSc-ILD over 52 weeks could not be developed using data from the overall placebo group. In this context, it is important to realise that the SENSCIS trial had broad inclusion criteria resulting in a heterogeneous patient population relatively representative of what is seen in clinical practice.20 33 We hypothesised that the difficulties in developing a strong model might be caused by the heterogeneity of the patient population: it could be that one risk model that fits all patients with SSc-ILD might be difficult to develop and that certain phenotypes/subgroups might need specific models. To test this hypothesis, we investigated whether a stronger model could be developed among patients with more homogeneous disease by developing a model in the subgroup of patients who had early and/or inflammatory SSc and/or severe skin fibrosis at baseline. This population is known to have rapid progression of SSc-ILD.24 34 In this subgroup, a stronger model for the prediction of progression of SSc-ILD was developed, with an apparent AUC of 0.75. Among the variables included in this model, a higher risk of progression was associated with lower DLco % predicted, shorter SSc duration, worse skin fibrosis, ATA positivity and not using mycophenolate at baseline. These findings are consistent with previous studies that have shown these factors to be associated with the progression of SSc-ILD.3 8 10 15 17 22 23 28
In the development of the prediction models, mycophenolate use was treated as a fixed (non-removable) variable. Evidence from randomised controlled trials supports the use of mycophenolate as a treatment for SSc-ILD,31 and its use in these patients has been recommended by Delphi consensus panels35 36 and in treatment guidelines recently issued by the American College of Rheumatology and American College of Chest Physicians37 and American Thoracic Society.38 While not a comparison of randomised groups, in the placebo group of the SENSCIS trial, the adjusted mean rate of decline in FVC over 52 weeks was 66.5 mL/year in patients taking mycophenolate at baseline and 119.3 mL/year in patients not taking mycophenolate.39
Higher mRSS and earlier SSc were included in the final risk model in the subgroup with early and/or inflammatory SSc and/or severe skin fibrosis at baseline, but not in the overall, more heterogeneous, placebo group. This might indicate that these factors are more important predictors of progression in this subgroup of patients than in the overall population. Another possible explanation is that, while the regression model estimates the average effect of a one-unit change in a variable, these two factors only affect the risk of progression when in the margins of their distributions; thus, as the subgroup of patients with early and/or inflammatory disease and/or severe skin fibrosis was enriched for patients with particularly high mRSS and/or early disease, we were better able to capture the relationship between these variables and the risk of progression in this subgroup.
We constructed a nomogram and provided formulas to enable practical use of the final prediction model in patients with early and/or inflammatory SSc and/or severe skin fibrosis at baseline. There was good agreement between the risk of progression predicted by this model and the observed risk of progression, indicating that the model is well calibrated. However, in the internal validation, the corrected performance measures were slightly worse than the non-corrected ones, reflecting a degree of overfitting. It is important to note that our nomogram only applies to patients with SSc-ILD who had risk factors related to early and/or inflammatory SSc and/or severe skin fibrosis at baseline. Further studies will be needed to develop risk prediction models in other subgroups of patients. Our model requires validation in different cohorts. Comparison of our model with other systems to predict the risk of SSc-ILD progression and mortality should be conducted in cohorts that are independent of those in which the models were developed.
Strengths of our analyses include the broad inclusion criteria and the robust collection of data in the setting of a clinical trial. However, the SENSCIS trial was not designed to assess factors associated with the progression of SSc-ILD, and these analyses should be regarded as exploratory. Assessment of progression was based only on decline in FVC and death and was limited to a period of 52 weeks. Other means of defining progression and the usefulness of the model for the prediction of SSc-ILD progression in the long term were not investigated. The potential impacts of symptoms, blood-based biomarkers and changes in clinical and radiological variables that may be associated with SSc-ILD progression were not explored. To avoid overfitting our model (given the limited sample size), we focused on a small number of factors, but we acknowledge that the factors leading to progression of SSc-ILD will be more complex.
In conclusion, SSc-ILD is a heterogeneous disease, and prediction of its progression may require different approaches in distinct subgroups of patients. Among patients in the SENSCIS trial who had early and/or inflammatory disease and severe skin fibrosis at baseline, the predictors of SSc-ILD progression over 52 weeks in a multivariable model were lower DLco % predicted, earlier SSc, worse skin fibrosis, ATA positivity and not using mycophenolate. A nomogram developed based on this model may be of value in the identification of patients in this subgroup of patients with SSc-ILD who are at risk of short-term progression.
Data availability statement
Data are available upon reasonable request. To ensure independent interpretation of clinical study results and enable authors to fulfil their role and obligations under the ICMJE criteria, Boehringer Ingelheim grants all authors access to relevant clinical study data. In adherence with the Boehringer Ingelheim Policy on Transparency and Publication of Clinical Study Data, scientific and medical researchers can request access to clinical study data, typically one year after the approval has been granted by major regulatory authorities or after termination of the development programme. Researchers should use https://vivli.org/ to request access to study data and visit https://www.mystudywindow.com/msw/datasharing for further information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by an independent ethics committee or institutional review board at every site. The SENSCIS trial was carried out in compliance with the protocol, the principles of the Declaration of Helsinki and the Harmonised Tripartite Guideline for Good Clinical Practice of the International Conference on Harmonisation. The participating sites are listed in the supplement to the primary manuscript.19 All patients provided written informed consent before trial entry.
Acknowledgments
The authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICMJE). The authors did not receive payment for the development of this manuscript. Elizabeth Ng and Wendy Morris of Fleishman-Hillard, London, UK, provided writing assistance, which was contracted and funded by Boehringer Ingelheim. Boehringer Ingelheim was given the opportunity to review the manuscript for medical and scientific accuracy as well as intellectual property considerations.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors MK, A-MH-V, GT, MA and OD were involved in the planning of this study. MK, A-MH-V and VS were involved in the acquisition of the data. GT led the data analysis. All authors were involved in the interpretation of the data and in drafting or revising the article. All authors approved the final version of the manuscript. MK is the guarantor for this manuscript.
Funding The SENSCIS trial was supported by Boehringer Ingelheim International GmbH.
Competing interests MK reports grants from Boehringer Ingelheim, Ono, MBL; consulting fees from Argenx, AstraZeneca, Boehringer Ingelheim, Chugai, GlaxoSmithKline, Kissei, Mochida; speaker fees from AbbVie, Asahi Kasei, Boehringer Ingelheim, Chugai, Eisai, Janssen, Ono; royalties/licenses from MBL. JA reports no disclosures. A-MH-V reports grants from Boehringer Ingelheim and Janssen; consulting fees from Arxx Therapeutics, Bristol-Myers Squibb, Boehringer Ingelheim, Genentech, Janssen, Medscape; speaker fees from Boehringer Ingelheim, Janssen, Medscape, Merck Sharp & Dohme, Roche; and support for attending meetings from Boehringer Ingelheim, Medscape, Roche; she is a EULAR study group leader on the lung in rheumatic and musculoskeletal diseases and a convener of the ERS/EULAR CTD-ILD guidelines. VS reports a grant from the Belgian Fund for Scientific Research in Rheumatic Diseases; research support from Boehringer Ingelheim; she is a senior clinical investigator of the Research Foundation—Flanders (Belgium) (FWO) (1.8.029.20N) (which had no involvement with the study reported in this manuscript). VS also reports consulting fees from Argenx BV, Boehringer Ingelheim, Janssen-Cilag, WebMD Global LLC; speaker fees from Boehringer Ingelheim, Galapagos, Janssen-Cilag; payment from BKC Moving Media Makers B.V.; support for attending congresses from Boehringer Ingelheim; participation on an advisory board and being an educational chair for Janssen-Cilag. VS has unpaid roles as chair of the EULAR study group on microcirculation in rheumatic diseases; as co-chair of the ACR study group on microcirculation and SCTC working group on capillaroscopy; and as a member of the Steering Committee of ERN-ReCONNET. GT and MA are employees of Boehringer Ingelheim. OD reports grants from Boehringer Ingelheim, Kymera, Mitsubishi Tanabe; consulting fees from 4P-Pharma, AbbVie, Acceleron, Alcimed, Altavant Sciences, Amgen, AnaMar, Arxx Therapeutics, AstraZeneca, Bayer, Blade Therapeutics, Boehringer Ingelheim, Corbus Pharmaceuticals, CSL Behring, Galapagos, Glenmark, Gossamer, Horizon Therapeutics, Janssen, Kymera, Lupin, Medscape, Merck, Miltenyi Biotec, Mitsubishi Tanabe, Novartis, Orion, Prometheus Biosciences, Redx Pharma, Roivant, Topadur, UCB; speaker fees from Bayer, Boehringer Ingelheim, Janssen, Medscape; and a patent issued for mir-29 for the treatment of systemic sclerosis (US8247389, EP2331143). OD also reports being Chair of the Executive Committee of the FOREUM Foundation, Co-Chair and President of the ERS/EULAR guidelines and EUSTAR, a member of the Board of Trustees of the Swiss Clinical Quality Management in Rheumatic Diseases, and the Hartmann Müller Foundation, a Senat member of the Swiss Academy of Medical Sciences, and Co-Founder of Citus AG.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.