Article Text

Download PDFPDF

Original research
Validation of three prediction models for thrombosis recurrence in antiphospholipid syndrome patients based on a prospective cohort
  1. Yuan Zhao,
  2. Can Huang,
  3. Wanting Qi,
  4. Yangzhong Zhou,
  5. Jiuliang Zhao,
  6. Qian Wang,
  7. Xinping Tian,
  8. Mengtao Li,
  9. Yan Zhao and
  10. Xiaofeng Zeng
  1. Department of Rheumatology and Clinical Immunology, Peking Union Medical College Hospital (PUMCH), Chinese Academy of Medical Sciences, Peking Union Medical College; National Clinical Research Center for Dermatologic and Immunologic Diseases (NCRC-DID), Ministry of Science & Technology; State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital; Key Laboratory of Rheumatology and Clinical Immunology, Ministry of Education, Beijing, China
  1. Correspondence to Professor Jiuliang Zhao; zjlpumc{at}sina.com; Professor Mengtao Li; mengtao.li{at}cstar.org.cn

Abstract

Objectives To validate the performance of the adjusted global antiphospholipid syndrome (APS) score (aGAPSS), Padua score and Caprini score to predict thrombosis recurrence in APS.

Methods Consecutive thrombotic-APS patients were included. aGAPSS, Padua and Caprini score at baseline were collected. Harrell c-index and calibration curve were used to validate the prediction models.

Results 362 patients were enrolled. The mean age was 36.30±13.88 years old, and 209 (57.7%) were female. Patients were followed up for a median of 2.32 years, with 32 (8.84%) venous and 21 (5.80%) arterial thrombosis. The 1-year, 3-year and 5-year thrombosis risks were 5.0%, 14.3% and 17.9%, respectively. The Harrell c-indexes of aGAPSS, Padua and Caprini score were 0.54 (95% CI 0.44 to 0.64), 0.54 (95% CI 0.46 to 0.62), and 0.50 (95%CI 0.42 to 0.58), respectively. Padua score had the best discrimination to predict venous thrombosis (Harrell c-index=0.61, 95% CI 0.53 to 0.69). aGAPSS had the best discrimination to predict arterial thrombosis (Harrell c-index=0.61, 95% CI 0.47 to 0.75). The calibrations for predicting thrombosis within 1, 3 and 5 years of the three models were suboptimal.

Conclusion The performance of aGAPSS, Padua and Caprini score to predict thrombosis recurrence in APS were suboptimal. Arterial and venous thrombosis recurrence predictors were different. New prediction models are required for venous and arterial thrombosis separately.

  • Antiphospholipid Syndrome
  • Lupus Erythematosus, Systemic
  • Thrombosis
  • Recurrence

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

What is already known on this topic

  • It is crucial to predict thrombosis recurrence in antiphospholipid syndrome (APS).

What this study adds

  • The discrimination and calibration of three models (adjusted global APS score, Padua, Caprini score) were suboptimal.

How this study might affect research, practice or policy

  • Venous and arterial thrombosis should be predicted separately in APS patients.

Introduction

Antiphospholipid syndrome (APS) is a rare and complicated acquired autoimmune thrombophilia characterised by arterial/venous thrombosis and/or recurrent pregnancy morbidity. The thrombotic recurrence rate of patients with persistent antiphospholipid antibody (aPL) profile in a cohort with a median of 172.5-month follow-up was 40.2%.1 The 10-year survival rate of APS patients is 90.7% and thrombosis is the first cause of death in APS patients,2 so early prediction of thrombosis and intensive anticoagulation treatment are crucial.

However, the thrombosis recurrence in APS patients is difficult to be predicted. The global APS score (GAPSS) and adjusted GAPSS (aGAPSS) were supposed to predict thrombosis recurrence, but they were developed for the prediction of thrombosis and pregnancy morbidity in systemic lupus erythematosus (SLE) patients based on a cross-sectional study,3 with and without anti-phosphatidylserine/prothrombin-complex antibodies; most validation studies only compared the baseline score between patients with and without thrombosis, but did not consider the influence of time.4–13 Other models that could be used to predict thrombosis included Padua score,14 which was used to predict venous thromboembolism in medical inpatients, and Caprini score,15 16 which was used to predict venous thromboembolism in surgical and medical patients. Both Padua score and Caprini score were widely used in clinical practice, but they were developed based on consensus approaches, and the predictive performance of these two models has not been validated in APS patients.

This study was conducted to validate the performance of aGAPSS, Padua score and Caprini score to predict thrombosis recurrence in our prospective APS cohort.

Methods

Study design

This was a single-centre prospective cohort study. Consecutive APS patients who fulfilled the 2006 Sydney Revised Classification Criteria for APS17 and had known thrombotic events at baseline, referred to Peking Union Medical College Hospital (PUMCH) from June 2012 to March 2022 were included. Baseline was defined as the time of APS diagnosis. Time to diagnosis was defined as the time between the first thrombosis and APS diagnosis. Follow-up time was defined as the time between APS diagnosis and the time of first thrombosis recurrence or the last follow-up visit. The overall follow-up ended in June 2022. Patients without follow-up visit were excluded.

The primary outcome was the first recurrence of venous or arterial thrombosis. Venous thrombosis included extremity deep venous thrombosis, pulmonary embolism, visceral venous thrombosis, cranial venous sinus thrombosis, jugular venous thrombosis, subclavian venous thrombosis and retinal venous thrombosis. Arterial thrombosis included ischaemic stroke, transient ischaemic attack (TIA), acute myocardial infarction, extremity arterial thrombosis, visceral arterial thrombosis, jugular arterial thrombosis, subclavian arterial thrombosis, vertebral arterial thrombosis and retinal arterial thrombosis.

Thromboses were confirmed by imaging and clinical diagnosis, such as ultrasound, CT angiography, MRI/magnetic resonance angiography, digital subtraction angiography, CT pulmonary angiography, ventilation/perfusion scan, ophthalmological examination and fluorescein angiography.

Data collection

Demographic characteristics were collected at baseline, such as age, sex, height and weight. Cardiovascular risk factors, including smoking history, hypertension, hyperlipidaemia were collected following National Institute for Health and Care Excellence guidelines.18 Thromboses were diagnosed according to the methods described earlier. Pregnancy morbidity was collected according to the 2006 Sydney Revised Classification Criteria for APS.17 Extra-criteria manifestations, including thrombocytopenia, haemolytic anaemia and microangiopathy were collected. Microangiopathy included APS-related nephropathy, livedo reticularis, heart valve disease, cognitive dysfunction, catastrophic APS and other microangiopathy confirmed by clinical diagnosis.19 Treatment regimen at baseline and at the time of thrombosis recurrence were collected.

aPLs included lupus anticoagulation (LA), anticardiolipin antibody (aCL) and anti-β2-glycoprotein I antibody (aβ2GPI). LA was measured according to the recommended three-step procedure with two test systems from the Scientific and Standardization Committee for lupus anticoagulant/aPLs of the International Society on Thrombosis and Hemostasis Subcommittee on lupus anticoagulant. LA was measured using activated partial thromboplastin time-based assay (aPTT) and the dilute Russell viper venom time (dRVVT), and the positivity was defined as aPTT ratio >1.20 or dRVVT ratio >1.20.20 aCL and aβ2GPI were measured by chemiluminescent immunoassay (CLIA) (iFlash CLIA kits provided by YHLO Biotech Co., Shenzhen, China). According to the manufacture’s instruction, the medium or high titre of aCL was defined as the titre >10 U/mL and the medium or high titre of aβ2GPI was defined as the titre >20 U/mL. This detection system showed good sensitivity and specificity in our cohort in the previous study.21 aPLs were considered positive only if confirmed at least 12 weeks apart.

aGAPSS,3 Padua score,14 Caprini score15 16 at baseline were calculated for all the patients. The predictors include aPLs, traditional cardiovascular risk factors, internal medicine diseases, surgery, trauma, malignancy, specific treatment regimen, etc. Definitions of the predictors are listed in online supplemental table 1. The definitions of acute risk situations in Padua score and Caprini score were the acute situations happened in 1 month before baseline. As the incidence rate of Factor V Leiden and Prothrombin 20 210A mutations are extremely low in Chinese patients,22 gene detection was not conducted with the consideration of costs.

Statistical analysis

Continuous variables were presented as means and SD for normally distributed data. Variables with abnormal distribution were presented as medians and IQRs (P25, P75). The Student’s t-test or Mann-Whitney U test was used for continuous variables. Categorical variables were presented as counts and percentages. Pearson χ2 test with continuity correction or Fisher’s exact test was used for categorical variables. The prognosis of thrombosis recurrence was shown by Kaplan-Meier curve. Since missing observations occurred for just two variables, missing values were predicted based on all other baseline characteristics with single imputation. The three models were validated based on Cox regression analysis (time-to-first-event outcomes were analysed using Cox regression analysis). Harrell c-index was used to evaluate the discrimination. As the baseline hazards of the three models were not reported, calibration could not be validated. We recalibrated the models in our cohort with aGAPSS, Padua score and Caprini score as linear predictors in Cox regression model, respectively.23 Therefore, in each prediction model, the baseline hazard was adjusted. Calibration curves were used to evaluate the calibrations of updated models.

Results

There were 481 patients in the APS cohort in PUMCH. 13 patients were lost during follow-up and 106 patients without known thrombosis history were excluded. A total of 362 thrombotic-APS (tAPS) patients were enrolled in this study (figure 1). The median time duration between diagnosis and enrolment was 1.25 months. The mean age was 36.30±13.88 years old on diagnosis, and 209 (57.7%) were female. The baseline characteristics in our cohort were listed in table 1, and were compared with those in the development cohort for aGAPSS (table 1). The development cohort for aGAPSS is an SLE cohort, with 100% patients diagnosed as SLE and only 42.2% patients diagnosed as APS. While the PUMCH validation cohort is an APS cohort, with only 23.5% patients diagnosed as SLE and 100% patients diagnosed as APS. In addition, the patients in PUMCH validation cohort had lower age (36.30±13.88) and less female (57.7%) than the patients in the development cohort for aGAPSS (42.6±12.1, 98.1%).

Table 1

Baseline characteristics of patients in this cohort compared with patients in development cohort for aGAPSS3

Figure 1

APS cohort in the database. APS, antiphospholipid syndrome; tAPS, thrombotic antiphospholipid syndrome.

Patients were followed up for a median of 2.32 years. During the follow-up period, there were 53 (14.6%) recurrent thrombotic events, 32 (8.84%) venous and 21 (5.80%) arterial thrombosis. None of the thrombotic events was fatal. The 1-year, 3-year and 5-year thrombosis risks were 5.0%, 14.3% and 17.9%, respectively. The 1-year, 3-year and 5-year venous thrombosis risks were 3.8%, 9.9% and 12.8%, respectively. The 1-year, 3-year and 5-year arterial thrombosis risks were 1.6%, 5.6% and 6.9%, respectively (figure 2). Among the 32 patients who developed venous thrombosis, 11 (34.4%) patients took warfarin, 4 patients took direct oral anticoagulants (DOAC) (12.5%) and 2 patients took aspirin (6.3%) regularly at the time of thrombosis diagnosis. Twelve (37.5%) patients took inadequate anticoagulation. The treatments of three (9.4%) patients were unknown. Among the 21 patients who developed arterial thrombosis, 7 (33.3%) patients took warfarin, 2 (9.5%) patients took DOAC and 1 (4.8%) patient took aspirin regularly at the time of thrombosis diagnosis. Ten (47.6%) patients took inadequate anticoagulation. The treatment of one (4.8%) patient was unknown (table 2).

Figure 2

Kaplan-Meier curves of venous, arterial and any thrombosis recurrence. (A) Any thrombosis recurrence; (B) venous and arterial thrombosis recurrence.

Table 2

The treatments at the time of thrombosis recurrence

Patients who developed venous thrombosis and patients who developed arterial thrombosis had different baseline characteristics. History of venous thrombosis was more common in patients who developed venous thrombosis. History of arterial thrombosis were more common in patients who developed arterial thrombosis (table 3). Patients secondary to SLE had different baseline characteristics compared with patients without SLE. They had younger age, shorter time to diagnosis and less smoking history. Female, hyperlipidaemia, microangiopathy and thrombocytopenia were more common in patients secondary to SLE (online supplemental table 2). In addition, patients with pregnancy morbidity had different baseline characteristics compared with patients with isolated tAPS. They had longer time to diagnosis, less smoking history, and more stroke/TIA and thrombocytopenia (online supplemental table 3).

Table 3

Different baseline characteristics between patients who developed venous thrombosis and patients who developed arterial thrombosis

The comparisons of baseline scores between each group were shown in figure 3. Patients who developed arterial thrombosis had higher aGAPSS than patients who did not (p=0.01). Patients who developed venous thrombosis had higher Padua score than patients who did not (p=0.03). The Harrell c-indexes for predicting thrombosis of aGAPSS, Padua score and Caprini score were 0.54 (95% CI 0.44 to 0.64), 0.54 (95% CI 0.46 to 0.62) and 0.50 (95%CI 0.42 to 0.58), respectively. The model predicting venous thrombosis with the best discrimination was Padua score (Harrell c-index=0.61, 95% CI 0.53 to 0.69), and the model predicting arterial thrombosis with the best discrimination was aGAPSS (Harrell c-index=0.61, 95% CI 0.47 to 0.75) (table 4). Online supplemental figure 1 demonstrated that sensitivity and specificity of three models were suboptimal. The calibration curves indicated that in most predictions except aGAPSS for arterial thrombosis in 5 years, the predicted thrombosis probabilities were higher than the actual thrombosis probabilities. Therefore, the calibrations for predicting thrombosis within 1, 3 and 5 years after diagnosis of all the three models were suboptimal (figure 4). As anticoagulation could influence the thrombosis recurrence, the discrimination and validation were also detected in the 231 patients with warfarin anticoagulation. The result was suboptimal either (online supplemental table 4, figure 2).

Table 4

Harrell c-index and 95% CI of each model

Figure 3

The comparison of baseline scores between patients with and without thrombosis recurrence. aGAPSS, adjusted global antiphospholipid syndrome score.

Figure 4

The calibration curves of three prediction models. (A, B, C) The calibration curve for venous thrombosis ((A) 1-year, (B) 3-year, (C) 5-year). (D, E, F) The calibration curve for arterial thrombosis ((D) 1-year, (E) 3-year, (F) 5-year). (G, H, I) The calibration curve for any thrombosis ((G) 1-year, (H) 3-year, (I) 5-year). aGAPSS, adjusted global antiphospholipid syndrome score.

Discussion

This is the first study validating the performance to predict thrombosis recurrence in APS patients of aGAPSS, Padua score and Caprini score based on a prospective cohort. We found that both the discrimination and calibration of these three models to predict venous, arterial and any thrombosis recurrence in APS patients were suboptimal. In addition, the performance to predict venous and arterial thrombosis of these three models were different, probably because patients who developed venous thrombosis and arterial thrombosis had different baseline characteristics. The probable reasons for the suboptimal predictive performance are listed below.

First, APS has a unique mechanism of thrombosis formation. The pathogenesis of thrombosis in APS patients is not exactly the same as the pathogenesis of thrombosis in other patients. Therefore, the predictive performance of Padua score and Caprini score are suboptimal. In addition, the prediction performance of these two models are not stable. Pandor et al demonstrated that Padua score and Caprini score had different predictive performances in different studies as well.24

Second, venous thrombosis and arterial thrombosis are two different clinical manifestations in APS patients with different pathogenesis. This study showed that history of venous thrombosis was more common in patients who developed venous thrombosis, while history of arterial thrombosis was more common in patients who developed arterial thrombosis. Previous studies found that venous thrombosis and arterial thrombosis had different risk factors in APS patients. Arterial thrombosis was associated with heart valve disease, hypertension, elevated anti-β2GPI IgM, history of arterial thrombosis, older age, smoking history and hyperhomocysteinaemia.12 25 26 Venous thrombosis was associated with history of venous thrombosis, heart valve disease and younger age.12 Apart from that, our study showed that aGAPSS had better performance to predict arterial thrombosis, which was consistent with previous results. Sciascia et al27 illustrated that the highest level of GAPSS was found in patients with arterial thrombosis in a pooled analysis. Radin et al6 also demonstrated that patients with recurrent arterial thrombosis but not venous thrombosis had higher aGAPSS. This is probably because hypertension is a more important risk factor for arterial thrombosis.12 25 26 In our study, 38.1% patients who developed arterial thrombosis had hypertension at baseline, while 21.9% patients who developed venous thrombosis had hypertension at baseline. Hypertension was more common in patients who developed arterial thrombosis, although the difference was not significant, which was probably because of the small sample size. Therefore, the risks of venous and arterial thrombosis recurrence in APS patients should be predicted separately.

Third, APS has a high degree of clinical heterogeneity. APS patients secondary to SLE have different characteristics compared with patients without SLE, as well as APS patients with pregnancy morbidity have different characteristics with isolated tAPS patients. Apart from that, both the use of glucocorticoid and high disease activity of SLE could increase the risk of thrombosis,28 29 which might also influence the predictive effect of the models. The high heterogeneity makes it difficult to predict thrombosis recurrence in APS patients.

Fourth, the influence of time and the difference between tAPS and obstetric-APS (oAPS) should be considered when developing and validating prediction models. GAPSS and aGAPSS have been the most widely used prediction model in APS patients until now, and many studies have confirmed good performance to predict thrombosis of GAPSS or aGAPSS. However, they were constructed to predict the risk of thrombosis and pregnancy morbidity in SLE patients based on a cross-sectional study,3 in which the influence of time was not considered and tAPS and oAPS patients were not distinguished. Some validation studies were conducted in aPLs positive patients10 or autoimmune diseases.13 One study was conducted in oAPS patients,9 and found that patients who developed thrombosis after pregnancy morbidity had higher baseline aGAPSS.9 One study enrolled 44 patients, finding that aGAPSS was higher in the 2 patients with recurrent thrombosis.11 Another study conducted in primary APS patients found that aGAPSS was higher in both patients with venous or arterial thrombosis recurrence.12 But most of the studies compared only the baseline aGAPSS between patients who did and did not develop thrombosis, without considering the influence of time. Some of them had a small size, and none of them compared the calibration of the model. Our previous study showed that tAPS and oAPS patients had different characteristics, and tAPS patients had much higher risk of recurrent thrombosis.30 However, there has been no study validating aGAPSS in tAPS patients. Our study validated the discrimination and calibration of aGAPSS in a tAPS cohort with 362 patients, and showed suboptimal performance of this model.

In this study, the sensitivity and specificity of aGAPSS were relatively suboptimal compared with other studies4 7 13 31 either. The reason might be the different study design and statistical analysis. The gold standard for receiver operator characteristic (ROC) curve of those studies were thrombosis history or thrombosis occurrence regardless of time, while we try to use baseline aGAPSS to predict thrombosis recurrence considering the recurrence time.

This study has several limitations. First, PUMCH is a tertiary hospital, patients in this centre are relatively more severe, and the clinical practices such as anticoagulation in PUMCH may differ from those in other centres. Second, the patients’ compliance to drugs, especially anticoagulation, could influence thrombosis recurrence. This study did not evaluate the influence of anticoagulation. We just validated the predictive ability of three models in the 231 patients with warfarin anticoagulation and found the suboptimal result. Third, due to the lack of detailed information of the three prediction models, the validation was based on recalibrating the models in our cohort. Fourth, multiple aPLs positivity is widely acknowledged as a risk factor of thrombosis recurrence,32 but it was not validated in our cohort because the number of events was not large enough to be further subgrouped. Multicentre studies with larger sample sizes are needed for further validation.

In conclusion, the discrimination and calibration of aGAPSS, Padua score and Caprini score to predict venous, arterial and any thrombosis recurrence in APS patients were suboptimal, probably because of the specificity and heterogeneity of APS. Patients who developed venous thrombosis and arterial thrombosis had different baseline characteristics. The construction of new prediction models respectively for venous and arterial thrombosis recurrence in APS patients is required to guide treatment.

Conclusion

The predictive performance of aGAPSS, Padua score and Caprini score to predict thrombosis recurrence in APS patients were suboptimal, probably because of the specificity and heterogeneity of APS. In particular, arterial and venous thrombosis recurrence predictors were different. New prediction models are required to guide treatment for venous and arterial thrombosis, respectively.

Data availability statement

Data are available upon reasonable request. The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics statements

Patient consent for publication

Ethics approval

This study involves human participants and was approved by the Medical Ethics Committee of Peking Union Medical College Hospital. Participants gave informed consent to participate in the study before taking part.

Acknowledgments

This study has been presented on EULAR 2022. Zhao Y, Huang C, Qi W, et al. Validation of 3 Prediction Models for Thrombosis in Antiphospholipid Syndrome Patients Based on a Prospective Cohort. Abstract for oral presentation. Annals of the Rheumatic Diseases, 2022; 81 (supplement 1): 93.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • YZ and CH are joint first authors.

  • Contributors YuanZ designed the study, analysed data, interpreted results and wrote the manuscript; CH interpreted results and revised the manuscript; WQ and YZhou analysed data, JZ designed the study, interpreted results, revised the manuscript, and guarantee the study, QW, XT, ML, YanZ and XZ supervised the study and revised the manuscript.

  • Funding This work was supported by the Chinese National Key Technology R&D Programme, Ministry of Science and Technology [2021YFC2501305 to JZ]; Beijing Municipal Science & Technology Commission [Z201100005520022, Z201100005520023, Z201100005520025, Z201100005520026, Z201100005520027 to XZ]; Chinese Academy of Medical Sciences Innovation Fund for Medical Sciences (CIFMS) [2021-I2M-1-005 to XZ.]; National High Level Hospital Clinical Research Funding [2022-PUMCH-A-008 to JZ, 2022-PUMCH-A-038 to CH]; and Municipal College Students’ Scientific Research Innovation Programme of Beijing [S202110023033 to YuanZ].

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.