Article Text
PDF
Abstract
Objective The 2019 European Alliance of Associations for Rheumatology (EULAR)/American College of Rheumatology (ACR) systemic lupus erythematosus (SLE) classification criteria score (≥20 points) has been associated with poor outcomes. We aimed to evaluate its utility as a predictor for mortality and hospitalisation and to derive and validate an ominosity score based on the SLE classification criteria set.
Methods Incident patients with SLE in a population-based cohort were included. The association between the 2019 EULAR/ACR SLE score and mortality and hospitalisation was assessed using Cox regression adjusted for age, sex and calendar year. An ominosity score for mortality was developed based on the SLE criteria set. The least absolute shrinkage and selection operator method was used to estimate model coefficients. Concordance and calibration were assessed by cross-validation and by plotting the observed event rates against the deciles of predicted probabilities.
Results Among 374 patients with incident SLE, a EULAR/ACR score ≥20 points was not associated with an increased risk of mortality (HR 1.17, 95% CI 0.67 to 2.03) or first hospitalisation (HR 1.14, 95% CI 0.79 to 1.64) compared with a score ≤19 points. The derived ominosity score for mortality included age, sex, thrombocytopaenia, neuropsychiatric manifestations, subacute cutaneous or discoid lupus, non-scarring alopecia, inflammatory arthritis, renal involvement, antiphospholipid antibodies and hypocomplementaemia. This model demonstrated a concordance=0.76 with adequate calibration. Age and sex were the main predictors, as seen in the model including just age, sex and year (concordance=0.77).
Conclusion The 2019 EULAR/ACR SLE criteria score was not associated with mortality and hospitalisation. The derived ominosity score for mortality presented good prediction for mortality but was not better than age and sex alone.
- Mortality
- Lupus Erythematosus, Systemic
- Epidemiology
- Outcome Assessment, Health Care
- Classification
Data availability statement
Data are available on reasonable request. Deidentified data are available after reasonable request and ethical approval.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
The 2019 European Alliance of Associations for Rheumatology (EULAR)/American College of Rheumatology (ACR) systemic lupus erythematosus (SLE) classification criteria have been tested for their association with disease activity, damage and mortality. It has been suggested that a score ≥20 points is associated with higher disease severity but their association with mortality has been inconsistent.
WHAT THIS STUDY ADDS
We aimed to evaluate the utility of the EULAR/ACR SLE classification criteria score to predict mortality and hospitalisation and to derive and validate an ominosity model based on the SLE classification criteria set.
The EULAR/ACR score was not associated with an increased risk of mortality or hospitalisation (neither dichotomised at 20 points or as a continuous score).
The derived ominosity model for mortality demonstrated a concordance of 0.76 with adequate calibration, but it did not perform better than the model that included age, sex and calendar year. The model for hospitalisation demonstrated poor performance, with a concordance of 0.52.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
We presented an ominosity model for predicting mortality with adequate performance; however, we were unable to derive a tool to predict hospitalisation. More work is needed to accurately predict mortality and hospitalisations among patients with SLE.
Introduction
Systemic lupus erythematosus (SLE) is an autoimmune disease characterised by a varied presentation that affects multiple organs and systems predominantly in women between puberty and menopause.1 Patients with SLE have a wide range of alterations in the immune system that lead to systemic inflammation and heterogeneous organ damage,2 contributing to excess mortality.3 Although the survival among patients with SLE has improved during the past decades,4 the mortality of patients with SLE is still almost three times higher than the general population.5 6 The clinical manifestations and complications due to SLE or its treatment can also lead to hospital admissions.7–9 Previous studies have demonstrated that patients with SLE are two times more likely to be hospitalised when compared with the general population.9 10 All these factors have an impact on quality of life, work productivity and healthcare cost; highlighting the importance of early identification of patients with SLE who have a higher risk of poor outcomes.
In 2019, the European Alliance of Associations for Rheumatology (EULAR) and the American College of Rheumatology (ACR) updated the classification criteria for SLE.11 These criteria and the resulting score aimed to identify patients for research studies.12 Nevertheless, they are widely used in clinical practice as guidance for SLE diagnosis. The EULAR/ACR SLE classification criteria have been tested for their association with disease activity,13–15 damage16–18 and mortality.16 17 It has been suggested that a score ≥20 points is associated with higher disease severity.14 However, the association between the 2019 EULAR/ACR SLE classification criteria score and mortality has been inconsistent,16 17 and other short-term outcomes (eg, hospitalisation) have not been explored. Additionally, the previous studies exploring these associations were performed in specialised centres, increasing the risk of overestimating poor outcomes and the susceptibility to referral bias by the enrichment of more severe SLE phenotypes.
We aimed to evaluate the utility of the 2019 EULAR/ACR SLE classification criteria score as a predictive tool for mortality and hospitalisation in a population-based cohort of patients with incident SLE. Second, we aimed to develop and validate an ominosity score to predict mortality and hospitalisations based on the 2019 EULAR/ACR SLE classification criteria. We hypothesised that a higher score in the SLE criteria would predict mortality and hospitalisations and that the derived tool would improve the performance to predict these poor outcomes compared with the current criteria set.
Methods
The Lupus Midwest Network (LUMEN) is a Centers for Disease Control and Prevention Lupus Registry nested within the Rochester Epidemiology Project (REP), a population-based medical records-linkage system from a 27-county region in southeast Minnesota and west central Wisconsin. The REP allows ready access to the medical records from healthcare providers for the local population, including institutions such as Mayo Clinic, Olmsted Medical Center, and their affiliated hospitals as well as local nursing homes, among others. The characteristics and strengths of the REP, as well as its generalisability, have been previously described.19 20 A detailed description of the LUMEN registry and the methodology used to identify, review and abstract data from potential SLE cases was provided elsewhere.6 10 21 We followed the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement for the reporting of this study.22
Study population and data collection
The study population included all residents who first fulfilled the 2019 EULAR/ACR SLE classification criteria within the timeframes of interest. For Olmsted County, we included all patients with incident SLE between 1 January 1976 and 31 December 2018. For the remaining 26 counties, we included all incident patients from 1 January 2010 to 31 December 2018. The EULAR/ACR SLE criteria components were abstracted by review of medical records. The date of first fulfilment of the criteria (ie, a score ≥10 points) was defined as the SLE incidence date. Given the retrospective nature of our data and to consider all the SLE manifestations that occurred early in the disease, we defined baseline for the purposes of this project as 6 months after SLE incidence date and used the 2019 EULAR/ACR SLE score at this time point in our analyses.
Mortality data were obtained from the REP for all patients since 1976. It incorporates data from the National Death Index and death certificates. Hospitalisation data were available starting in 1995 and electronically retrieved; only incident patients after this time point were included in the hospitalisation analyses. Patients were followed until the last follow-up, death or 31 March 2022.
Statistical analysis
Descriptive statistics were used to summarise the patient demographics and clinical characteristics. Cox proportional hazards models were used to examine the association between the 2019 EULAR/ACR score (per 4-point increase and dichotomised as ≥20 points vs <20 points) at baseline and the outcomes of mortality and either first hospitalisation or death after SLE incidence, adjusted for age, sex and calendar year of SLE incidence. Patients alive at last follow-up were censored for both, mortality and hospitalisation. The proportional hazards assumption was examined using the Schoenfeld residuals, and no violations of the proportional hazards assumption were found. Non-linear effects of the EULAR/ACR score on each outcome were assessed using penalised smoothing splines and the fit was plotted to visually assess the potential utility of a cut-off score. Sensitivity analysis with the EULAR/ACR score 12 months after SLE incidence was also performed.
Associations between individual items from the 2019 EULAR/ACR SLE classification criteria and outcomes were assessed using multivariable Cox proportional hazards models adjusted for age, sex and index year. HR estimates and 95% CIs were presented. Individual parameters that were clinically redundant (ie, proteinuria and biopsy-proven nephritis) or too rare to consider individually (ie, delirium, psychosis and seizures), were combined under a common domain to simplify and reduce the number of covariates in the models. The least absolute shrinkage and selection operator method was used for model development. Fivefold cross-validation was used to select the penalty which minimised validation loss. To avoid overoptimistic estimates, nested fivefold cross-validation was used to estimate concordance (which is analogous to the area under the receiver-operating characteristic curve for binary outcomes). Both apparent (optimistic) and cross-validated estimates for model concordance (ie, discrimination, or the ability to correctly rank patients from low to high risk) were presented. To assess model calibration (ie, the accuracy of the risk estimates), the cohort was divided into 10 deciles of predicted probability of the given event (eg, death at or prior to 5 years), with Kaplan-Meier estimates (95% CI) for the event plotted against the mean predicted probabilities within decile group. Baseline survival at specified time points was estimated using the final predictive models assuming index year 2018.23 P values <0.05 were considered statistically significant. A summary of data analysis steps is represented in figure 1. Analyses were performed using SAS V.9.4 (SAS Institute) and R V.4.1.2 (R Foundation for Statistical Computing).
Data analysis steps to assess the EULAR/ACR SLE classification score and derive and cross-validate the ominosity score. ACR, American College of Rheumatology; EULAR, European Alliance of Associations for Rheumatology; LASSO, least absolute shrinkage and selection operator; LUMEN, Lupus Midwest Network; SLE, systemic lupus erythematosus.
Results
We studied 374 patients with incident SLE. The median (IQR) age at SLE incidence was 46 (32–61) years; 80% were female and 82% were non-Hispanic white. The median length of follow-up was 8 (IQR 6–13) years. The most common clinical manifestations at baseline (ie, 6 months after SLE incidence) were inflammatory arthritis (59%), leucopenia (41%) and acute cutaneous lupus (24%). 69% were anti-dsDNA positive and 13% had proteinuria (online supplemental table 1). During follow-up, 91 patients died. Among the 314 patients with hospitalisation data (after 1995), 170 experienced at least one hospitalisation. The median EULAR/ACR score was 12 (IQR 12–15) points at SLE incidence and 14 (IQR 12–19) points at baseline. 82 (23%) patients had a score of 20 or more points at baseline.
Supplemental material
EULAR/ACR SLE classification score, mortality and hospitalisation
Among patients with incident SLE, a EULAR/ACR score ≥20 points at baseline was not associated with an increased risk of mortality (HR 1.17, 95% CI 0.67 to 2.03) or first hospitalisation (HR 1.14, 95% CI 0.79 to 1.64) compared with a score ≤19 points (table 1). When assessing the EULAR/ACR score as a continuous variable, there was no statistically significant association with mortality (HR 1.04 per 4-point increase, 95% CI 0.88 to 1.23), nor with hospitalisation (HR 1.08 per 4-point increase, 95% CI 0.97 to 1.21; table 1). The sensitivity analyses with the EULAR/ACR score 12 months after SLE incidence were consistent; except for hospitalisation when used as a continuous variable, which was associated with a 12% increased risk, per 4-point increase in score (HR 1.12, 95% CI 1.01 to 1.24, online supplemental table 2). We did not find evidence of non-linear association between the EULAR/ACR score and mortality (p=0.081) or hospitalisations (p=0.104), nor did we see visual evidence that suggested another potential cut-off score (figure 2).
Estimated functional relationship between the EULAR/ACR SLE classification score and the hazard for mortality (A) and hospitalisation (B). ACR, American College of Rheumatology; EULAR, European Alliance of Associations for Rheumatology; SLE, systemic lupus erythematosus.
Association between the 2019 EULAR/ACR score at baseline (ie, 6 months after systemic lupus erythematosus (SLE) incidence) and the outcomes of mortality and hospitalisation among patients with SLE in a population-based cohort
Association between the components of the EULAR/ACR SLE classification criteria and mortality or hospitalisation
In the analysis of the individual components of the EULAR/ACR SLE classification criteria, the presence of neuropsychiatric (HR 4.22, 95% CI 1.52 to 11.72) and renal (HR 2.18, 95% CI 1.22 to 3.89) manifestations at baseline, along with the positivity to antiphospholipid antibodies (HR 1.89, 95% CI 1.10 to 3.27) and having low levels of both C3 and C4 complements (HR 1.87, 95% CI 1.12 to 3.14) were associated with higher mortality; presenting inflammatory arthritis (HR 0.40, 95% CI 0.25 to 0.63) was associated with lower mortality after adjusting for age, sex and calendar year (table 2).
EULAR/ACR SLE classification criteria components at baseline (ie, 6 months after systemic lupus erythematosus (SLE) incidence) and their association with mortality and hospitalisation
We observed that the presence of neuropsychiatric manifestations (HR 4.90, 95% CI 2.26 to 10.60), serosal involvement (HR 1.76, 95% CI 1.24 to 2.49) and low levels of both C3 and C4 (HR 1.45, 95% CI 1.06 to 2.00) were associated with a higher risk of hospitalisation. Of note, inflammatory arthritis (HR 0.74, 95% CI 0.56 to 0.97) was associated with lower risk of hospitalisation (table 2).
Development of ominosity score
The fully adjusted model for predicting mortality included age (per 10 years), sex, thrombocytopaenia, neuropsychiatric manifestations, subacute cutaneous or discoid lupus, non-scarring alopecia, inflammatory arthritis, renal involvement, antiphospholipid antibodies and hypocomplementaemia. The model for hospitalisation included sex, thrombocytopaenia, autoimmune haemolysis, neuropsychiatric manifestations, oral ulcers or acute lupus, serosal involvement and inflammatory arthritis (table 3).
EULAR/ACR SLE classification components and estimated coefficients for prediction of mortality and hospitalisation
Performance of ominosity score
The apparent (optimistic) concordance of the ominosity score for mortality was 0.81, while concordance estimated by cross-validation was 0.76. This concordance was not better than the model with just age, sex and calendar year (concordance=0.77). The calibration of the model was fair as can be seen in the calibration plots of predicted risk and their 95% CI at 5 and 10 years after baseline (online supplemental figure 1A,B).
The model for hospitalisation demonstrated poor performance (optimistic concordance 0.63, cross-validation concordance 0.52; calibration plots in online supplemental figure 2A,B), comparable to the model with just age, sex and calendar year (concordance=0.56).
Discussion
This study evaluated the association of the current 2019 EULAR/ACR SLE classification criteria with the outcomes of mortality and hospitalisation in a population-based cohort of incident patients with SLE. We found that the EULAR/ACR SLE score was not associated with mortality or hospitalisation either testing a threshold of 20 points or using the score as a continuous variable. Subsequently, we developed a predictive model based on the components of the EULAR/ACR SLE criteria set. We found that the model was able to predict mortality with adequate performance but did not perform well for predicting hospitalisation. Importantly, the mortality model did not outperform age and sex alone.
Since the publication of the EULAR/ACR criteria and score for SLE classification in 2019, there have been several attempts to assess their association with poor outcomes. We did not find association among patients from a population-based SLE cohort between the original EULAR/ACR score and mortality. The first study that assessed the association between the EULAR/ACR score at clinical diagnosis of SLE and mortality was from the University College Hospital in London.16 Although they did not find any association, their analysis had some limitations: their cohort was susceptible to referral and selection bias, which was reflected by an enriched study population of patients with higher EULAR/ACR scores at diagnosis (median of 24 points), and the lower mortality rate at ten years (7%) compared with our study. Both risks of bias were overcome by our population-based design. Another study from the University of Toronto lupus cohort with 867 patients described a higher mortality among patients with a score ≥20 points (9.7%) against those with <20 points (5.8%), with an HR=2.34 for the higher score group compared with the lower score group.17 There were several differences that might explain the contrasting findings with our study. Similar to the study from London, the Toronto cohort was susceptible to referral bias, potentially enriching the high score group. Also, it was unclear how the vital status of the patients in the cohort was ascertained across the years of follow-up. It was interesting that the Toronto study did not adjust for sex in their regression analyses, which has been linked with differential mortality risk in SLE.24 Considering the previous studies and our unsuccessful attempt to find another valid threshold, we consider that the utility of the original EULAR/ACR SLE score for predicting mortality in the community setting is limited.
The observed lack of utility of the original EULAR/ACR SLE score for predicting mortality motivated us to pursuing an improved score system framed on the same domains and items included in the EULAR/ACR SLE criteria—keeping the same applicability—but tailored to the individual contribution of their components. We found that age, thrombocytopaenia, neuropsychiatric manifestations, subacute or discoid lupus, renal involvement, antiphospholipid antibodies positivity and hypocomplementaemia (for both C3 and C4) were associated with an increased risk of mortality. In contrast, female sex, non-scaring alopecia and inflammatory arthritis were negatively associated with mortality. Many of the included predictors were associated independently in previous studies. Our study assessed the complete set of criteria in a single tool and considered positive and negative predictors; this approach allows a more precise and personalised prognosis. The derived ominosity model, as proposed, demonstrated fair-to-good performance during cross-validation in this study but did not outperform a model that included only age and sex.
Unfortunately, we were unable to derive an adequate model for predicting hospitalisation. Our derived model (and the original EULAR/ACR score) performed poorly to predict a first hospitalisation in patients with new-onset SLE. Even though we found some components associated with an increased risk of hospitalisation (eg, neuropsychiatric manifestations, serosal involvement and hypocomplementaemia); the whole EULAR/ACR SLE criteria were not helpful to discriminate the patients more likely to experience a hospitalisation. This might be explained partially by the nature of several domains included in the criteria set. For example, very rarely a patient with cutaneous involvement (particularly those included in the criteria), arthritis or isolated laboratory abnormalities (eg, leucopenia, positive SLE antibodies or hypocomplementaemia) will require a hospitalisation. Although some parameters such as serositis, neuropsychiatric manifestations, severe lupus nephritis or when the abnormal laboratories indicate severe disease (like hypocomplementaemia or profound cytopenias) may require and justify hospitalisation, SLE activity is not the only nor the most common reason for hospitalisation, and these predictors might be unrelated to other common reasons for hospitalisation like infections or cardiovascular diseases.8 10 25 26
The limitations of our study included a modest sample size for deriving the risk prediction model and the lack of external validation for the derived ominosity score. However, we used fivefold cross-validation as an alternative. The derivation population is predominantly non-Hispanic white and our results may not generalise to more diverse populations. Some of the predictors had low prevalence or were rarely observed, giving some imprecision to the estimated coefficients. It would be interesting to assess in the future if the accumulation of additional manifestations later in the course of the disease might modify the predicted risk. Although we did identify components with lower or higher risk for poor outcomes, this is limited to 2019 EULAR/ACR SLE criteria components, and other factors that might impact outcomes were not considered, including those that could happen beyond the first 6–12 months of the disease. This may warrant further refinement of ominosity score.
Strengths of this study include the population-based setting, covering the complete clinical spectrum of the disease (from mild to severe) across all levels of care, avoiding the risk of referral bias and deriving a predictive model from an enriched severe cases population. The ascertainment of our outcomes comes from the REP, which links data across all the public and private providers within the coverage area and not only relies on local or health-system-derived data, and for the case of deaths, it incorporates data from the National Death Index and death certificates, assuring adequate measurement of the outcome. The LUMEN cohort included patients of all ages, permitting the applicability of the ominosity score to any age. Although extremely young and elderly ages are uncommon in the cohort, this is reflective of the epidemiology of the disease. We kept the original domains and items in the SLE criteria to maintain their simultaneous applicability, only merging those manifestations clinically related in an attempt to get a parsimonious and coherent model. We incorporated age and sex in the model, which are known predictors of mortality and hospitalisation, and shown to be important predictors in our analyses.
In conclusion, the original EULAR/ACR SLE score was not associated with mortality or hospitalisation. We presented an ominosity model for predicting mortality with adequate performance; however, we were unable to derive a tool to predict hospitalisation. More work is needed to accurately predict mortality and hospitalisations among patients with SLE.
Data availability statement
Data are available on reasonable request. Deidentified data are available after reasonable request and ethical approval.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by the institutional review boards of Mayo Clinic (20-006485) and Olmsted Medical Center (036-OMC-20).
Acknowledgments
This study is derived from Gabriel Figueroa-Parra’s final thesis as a partial requirement to obtain his master’s degree in clinical and translational science. He would like to thank to Matthew J. Koster, MD, Ladan Zand, MD, and Manuel F. Ugarte-Gil, MD, MSc, members of his thesis advisory committee.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
AD-G and CC are joint senior authors.
X @DrGabrielFP, @CrowsonCindy
AD-G and CC contributed equally.
Presented at Preliminary results were presented at ACR Convergence 2023 (Figueroa-Parra G, Hanson A, Sanchez-Rodriguez A, Meade-Aguilar J, Crowson C, Duarte-Garcia A. Utility of the 2019 EULAR/ACR SLE Classification Criteria Score as Predictor for Mortality and Hospitalizations in a Population-Based Cohort: The Lupus Midwest Network (abstract). Arthritis Rheumatol. 2023; 75 (suppl 9). https://acrabstracts.org/abstract/utility-of-the-2019-eular-acr-sle-classification-criteria-score-as-predictor-for-mortality-and-hospitalizations-in-a-population-based-cohort-the-lupus-midwest-network/).
Contributors GF-P, ACH, AD-G and CC contributed to the study conception and design. Material preparation and data collection were performed by ACH, AD-G and CC. Analyses of data were performed by ACH and CC. Interpretation of results was made by GF-P, ACH, AS-R, JAM-A, MG-T, MCC-G, KEB, AD-G and CC. AD-G and CC verified the underlying data and are guarantors of the study. The first draft of the manuscript was written by GF-P. All authors read and approved the final manuscript.
Funding The Lupus Midwest Network (LUMEN) is supported by the Centers for Disease Control and Prevention (CDC) of the US Department of Health and Human Services (HHS) under Grant number U01 DP006491 as part of a financial assistance award totalling US$1 750 000 with 100% funded by CDC/HHS.
Disclaimer This study used the resources of the Rochester Epidemiology Project (REP) medical records-linkage system, which is supported by the National Institute on Aging (AG 058738), by the Mayo Clinic Research Committee, and by fees paid annually by REP users; and Grant number UL1 TR002377 from the National Center for Advancing Translational Sciences, a component of the National Institutes of Health. The content is solely the authors’ responsibility and does not necessarily represent the official views of the National Institutes of Health. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC.
Competing interests AD-G is supported by the Rheumatology Research Foundation Investigator Award and the Lupus Research Alliance Diversity in Lupus Research Award. The rest of the authors have nothing to disclose.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.