Article Text
Abstract
Objectives This study aims to investigate COVID-19 epidemiological data in patients with autoimmune inflammatory rheumatic diseases (AIRDs) during Omicron wave and to identify clinical factors associated with infection, including COVID-19 vaccination.
Methods This prospective longitudinal study was performed between January and October 2022 in South Korea. Patients were classified into AIRD and non-AIRD groups according to their underlying diseases. COVID-19 status, date of confirmed infection and vaccination status were captured from the patient survey and national database. The COVID-19 incidence during the study period was examined and compared between the two groups. The effect of clinical factors on the infection rate was analysed in the AIRD group.
Results A total of 1814 patients (1535 and 279 in the AIRD and non-AIRD groups, respectively) were analysed. During the study period, 857 COVID-19 cases were reported in 834 patients (46.0%). The infection rates in the AIRD and non-AIRD groups were comparable. In the AIRD group, older age (≥70 years) and glucocorticoid use were significantly associated with a lower rate of COVID-19 infection. The third booster vaccination significantly lowered the incidence of COVID-19 (adjusted HR 0.85 (95% CI 0.73 to 0.99)), and the prophylactic effect was more evident in patients aged <70 years (0.81 (95% CI 0.69 to 0.95), p value for interaction 0.036).
Conclusion The risk of SARS-CoV-2 infection with the Omicron variant did not increase in patients with AIRDs. The third booster vaccination regimen decreased the infection rate in patients aged <70 years.
- Infections
- COVID-19
- Vaccination
- Epidemiology
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Before the Omicron outbreak, certain patients with autoimmune inflammatory rheumatic diseases (AIRDs) might be more vulnerable to COVID-19 infection than the general population.
Patients with AIRDs generate lower antibody response to COVID-19 vaccination than in the general population.
WHAT THIS STUDY ADDS
During the Omicron era, the risk of COVID-19 infection was not increased in patients with AIRDs.
The presence of AIRDs, comorbidities and immunosuppressants are not significantly associated with higher incidence after the Omicron outbreak.
The third booster vaccination significantly reduces the incidence of COVID-19 in patients with AIRDs, particularly in those aged <70 years.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Patients with AIRDs should be advised to continue their treatment unchanged during the Omicron era.
The third booster vaccination should be universally considered in patients with AIRDs to reduce the risk of future COVID-19 infection.
Introduction
COVID-19 posed the greatest challenge to human health worldwide. Despite efforts to control this disease, several new variants have emerged, altering its characteristics and making disease control more difficult. In late 2021, the Omicron variant (BA.1(B.1.1.529.x), BA.2) was first identified in South Africa. Due to its high transmission rate, it has become the most dominant variant in the world since January 2022, as well as in South Korea.1
South Korea has unique national strategies against the COVID-19 pandemic. Since February 2020, South Korea has implemented active preventive measures, including mandatory mask wearing, strict isolation of affected patients and a national vaccination project. These measures helped control the spread of the virus until the end of 2021.2 However, after the Omicron outbreak, daily cases of COVID-19 in South Korea increased steeply, reaching >600 000 cases/day, which was the highest incidence rate in the world at that time. As a result, the Omicron variant has had a significant impact on both the general population and national healthcare system.
Before the Omicron era, several studies investigated the incidence of (severe) COVID-19 in patients with autoimmune inflammatory rheumatic diseases (AIRDs).3–15 In general, most studies have shown similar incidence rates compared with the general population4 5 15 while others have shown increased infection in certain subpopulations of AIRDs or certain patients taking specific antirheumatic medications.3 9 10 12 14 16 In a meta-analysis of 23 studies published in 2021, the infection rate of COVID-19 in patients with AIRDs was significantly higher than that in the general population (risk ratio 1.53).12 Furthermore, some studies reported higher incidence of severe COVID-19 outcome in patients with AIRDs compared with the general population.6–8 10 11 A multicentre cohort study performed before the Omicron era showed that the hospitalisation rate of COVID-19 patients with AIRDs was 14% higher than that of matched controls.8 However, the Omicron variant is completely different from previous variants in virology. Recent studies have indicated that this variant exhibits a distinct viral entry pathway and a high number of mutations, which contribute to its heightened transmissibility, potent immune evasion capacity and relatively low pathogenicity.17 18 However, there is a lack of epidemiological data regarding the impact of the Omicron variant of COVID-19 on patients with AIRDs. This study aimed to investigate the incidence of COVID-19 and the impact of clinical factors, including COVID-19 vaccination, on the infection rate and outcomes of patients with AIRDs during the Omicron outbreak.
Methods
Study design and population
This prospective longitudinal study was performed during the Omicron era in South Korea (1 January 2022 to 31 October 2022). We included patients who visited the rheumatology outpatient clinic of the Seoul National University Hospital (SNUH), one of the largest nationwide tertiary referral centres in South Korea, from 1 January 2022 to 30 April 2022. The included patients were followed up every 1–3 months as part of their daily clinical practice and participated in a survey regarding COVID-19 infection and history of COVID-19 vaccination during the period between 17 August and 8 November. The index date was defined as 1 January 2022. To detect all COVID-19 infection occurred in 2022, observation and monitoring for COVID-19 infection started on 1 January 2022 for all included patients. Observation period was terminated on the date of participating survey.
We classified the included patients into two groups based on their underlying rheumatic diseases (AIRD vs non-AIRD). We assigned their rheumatic diseases by their primary diagnosis. The primary diagnosis was defined as the primary reason for antirheumatic medication use on the index date. The International Classification of Diseases 10th Revision (ICD-10) codes for the diseases included in the AIRD are presented in the ‘online supplemental text S1’. Patients in the non-AIRD group were followed up for non-inflammatory, non-autoimmune diseases (osteoarthritis, fibromyalgia, gout without flares and primary Raynaud syndrome) and were not prescribed any antirheumatic agents except on-demand, short-term glucocorticoids (GCs) for acute gout flares. We excluded patients who were under 18 years old, who had active cancer or a history of transplantation, or who were infected with COVID-19 before 1 January 2022. Patients with AIRDs who followed up our rheumatology clinic for less than 6 months at the index date were also excluded because they had relatively short duration of medication and diagnosis of specific AIRD was not confirmed in some patients.
Supplemental material
Data collection
Information on demographics, comorbidities and medications was collected from the electronic medical records database of SNUH (SUPREME). Comorbidities such as hypertension, diabetes mellitus, obesity, chronic kidney disease (CKD) and chronic liver disease were identified by reviewing the patients’ prescription data and medical records. Concomitant antirheumatic agents were defined as those prescribed within 3 months prior to the index date. They were classified into immunomodulators (conventional/biologic/targeted-synthetic disease-modifying antirheumatic drugs (DMARDs)) and immunosuppressants except GCs (mycophenolate mofetil, azathioprine, calcineurin inhibitor, cyclophosphamide) and GCs as recommend by Isaacs and Burmester.19 All the laboratory findings and disease activity as per the physician’s global assessment (low disease activity vs moderate or severe disease activity) of the included patients were collected during the study period. The patients’ personal information was encrypted, and all processes were run by the Information Security team at SNUH. To compare the incidence rate of COVID-19 during the study period with that in the general population, we collected public epidemiological data (including COVID-19 infection, vaccination and hospitalisation rates) from the Korea Disease Control and Prevention Agency (KDCA), which was updated daily during the study period. The national surveillance system for COVID-19 during the study period and the process for KDCA registration were described in the ‘online supplemental text S2’.
Patient survey
The included patients participated in a survey on COVID-19 infection and vaccination between 17 August 2022 and 8 November 2022. The survey period was selected considering that most of the included patients were followed up every 12 weeks. In addition, because a patient infected with COVID-19 should undergo isolation for 7 days according to domestic regulations, the investigation period was extended by 8 days to capture the COVID-19 infection that occurred in late October.
The questionnaire included COVID-19 infection status, date of confirmed diagnosis of COVID-19 based on PCR or rapid antigen test, COVID-19-related hospitalisation and adjustment of medication for the underlying disease during COVID-19 infection. In South Korea, COVID-19 was confirmed using PCR only until March 2022. Subsequently, the rapid antigen test was approved as an alternative method for a confirmative diagnosis. To minimise recall bias, the date of the COVID-19 infection was validated using smartphone messages from patients with confirmed infection from the KDCA.
The vaccination history for COVID-19 was also obtained using the survey and participants’ mobile vaccination certificates. The vaccination status on the index date was defined as the participant’s baseline vaccination status. We categorised baseline vaccination status into (1) unvaccinated, (2) first vaccinated and second vaccinated and (3) third-booster vaccinated. Baseline vaccination types were classified as (1) mRNA (BNT162b2 and mRNA-1273), (2) non-mRNA (ChAdOx1 nCoV-19, Ad26.COV2.S and NVX-CoV2373) or (3) mixed vaccine (cross-vaccination). Since mRNA vaccines were recommended as a third dose of booster vaccination in whole people except who have contraindications, there was no participant who received the non-mRNA vaccine as a third dose of booster vaccination on the index date.
Outcome
The primary outcome of this study was the incidence of COVID-19 during the study period. Cumulative infection rate of COVID-19 was estimated weekly and compared between the AIRD and non-AIRD groups. Secondary outcomes included the incidence rate of hospitalised COVID-19 patients and the clinical factors associated with COVID-19. The effects of vaccination on the incidence of COVID-19 were also investigated.
Statistical analysis
All analyses were performed using observational data, and missing data were not imputed. χ2 and Student’s t-tests were used to compare the categorical and continuous variables between the two groups, respectively. The risk factors for COVID-19 infection were investigated using the Cox proportional hazards model. Covariates that showed a relevant association (p<0.1) with the outcome in the univariable analysis were included in the multivariable analysis. Vaccination status at the index date was used as a covariate. In addition, we performed a cluster Cox regression to consider the effect of subsequent vaccination after the index date on the infection rate of COVID-19. Statistical analyses were performed using R (PBC, Boston, USA, V.4.1.1) and a p<0.05 was considered statistically significant.
Results
Patient characteristics
A total of 2050 patients were included in this study; of these, 1814 were analysed, comprising 1535 patients diagnosed with AIRD (AIRD group) and 279 patients without AIRD (non-AIRD group) (online supplemental figure 1). The mean (SD) follow-up duration was 202.4 (89.4) days.
The baseline characteristics of the included patients are presented in table 1. Patients in the AIRD group were younger (56.4 vs 58.9 years) and less likely to have comorbidities, such as diabetes and CKD, than those in the non-AIRD group. Overall, 155 (8.4%) patients did not receive COVID-19 vaccination at the index date, with a significantly higher prevalence observed in the AIRD group (9.2% vs 5.0%, p=0.030). The proportion of patients who received a third dose of booster vaccination was comparable between the two groups (51.5% vs 56.6%). In both groups, older (≥70 years) patients showed higher prevalence of the third booster vaccination because the third vaccination was performed in patients older than 70 years by priority from October 2021 in South Korea.
Baseline characteristics of the patients
The clinical features of participants in the AIRD group are summarised in online supplemental table 1. Rheumatoid arthritis (RA) was the most common underlying disease (n=516, 33.6%), followed by spondyloarthritis (n=388, 25.3%), and systemic lupus erythematosus (n=157, 10.2%). A total of 623 (40.6%) patients had received GCs, with a mean (SD) dose of 5.3 (5.6) mg/day of prednisone or its equivalent. Additionally, 1230 (80.1%) were treated with at least one antirheumatic agents.
Incidence of COVID-19 infection and its outcome
During the study period, 834 (46.0%) patients reported 857 COVID-19 infections (including reinfection), with an incidence rate (per 1000 person-years) of 553.8 (95% CI 516.8 to 592.7). The infection rate in the general population of South Korea was 43.9% during the study period. The longitudinal change in COVID-19 incidence showed two peaks in March and August, which was consistent with that in the general population of South Korea (figure 1A). When the incidence was stratified by age, it decreased with increasing age (figure 1B). The infection rate of COVID-19 in the AIRD group was not significantly different from that in the non-AIRD group (46.3% vs 44.4%, p=0.622). This trend was consistent in most age groups, although the incidence in those aged between 20 and 29 years was numerically higher in the AIRD group (70.5% vs 53.3%, p=0.316) (online supplemental table 2).
COVID-19 infection rate in the study population during the study period. (A) Cumulative incidence rate of COVID-19 in the study population and (B) overall infection rate of COVID-19 stratified by age. AIRD, autoimmune inflammatory rheumatic disease.
The clinical manifestations of patients with COVID-19 are presented in online supplemental table 3. Most patients showed non-severe manifestations and 78 (9.4%) were asymptomatic. No significant differences were observed in the clinical features of COVID-19 between the AIRD and non-AIRD groups. In contrast to the high incidence rate of COVID-19, only 30 cases of hospitalisation caused by COVID-19 occurred during the study period. The rate of hospitalisation among the infected patients was also comparable between the two groups (3.8% vs 3.4%, p=0.847).
Risk factors for COVID-19 infection in patients with AIRD
In the AIRD group, univariable analysis showed that older age (≥70 years) was significantly associated with lower incidence of COVID-19 (unadjusted HR (HR) 0.61 (95% CI 0.49 to 0.76)). Patients with comorbidities such as hypertension and CKD also showed similar effects. In contrast, patients with spondyloarthritis showed significantly higher COVID-19 incidence compared with the other study population (HR 1.20 (95% CI 1.02 to 1.42)). Regarding the treatment of the underlying AIRD, patients treated with GCs had a lower risk of COVID-19, whereas treatment with other immunomodulators or immunosuppressants was not a risk factor (table 2). There was still no significant association between antirheumatic medications and infection, except GCs, when we categorised medications into monotherapy or combination therapy (online supplemental table 4).
Risk factors for COVID-19 infection in patients with AIRD
In the multivariable analysis that included relevant clinical factors as covariates, older age (adjusted HR 0.70 (95% CI 0.56 to 0.88)) and concomitant GC treatment (adjusted HR 0.84 (95% CI 0.71 to 0.998)) were significantly associated with lower incidence of COVID-19 infection.
Effect of vaccination on COVID-19 infection in patients with AIRDs
During the study period, at least one vaccination did not significantly decrease the incidence of COVID-19 (HR 1.06 (95% CI 0.82 to 1.38)) in the AIRD group (figure 2A). This result was consistent irrespective of the presence of clinical factors. In contrast, patients who received the third booster vaccination showed a significantly lower incidence of COVID-19 infection compared with other patients (HR 0.79 (95% CI 0.68 to 0.92)) (figure 2B). This result was also consistent in the multivariable analysis adjusted for patient age and GC use (adjusted HR 0.85 (95% CI 0.73 to 0.99)). The protective effect of the booster vaccination was also consistent in the non-AIRD population (online supplemental figure 2).
Kaplan-Meier curves indicating the effect of vaccination on COVID-19 incidence during the study period. (A) Effect of any dose of vaccination. (B) Effect of the third dose of booster vaccination.
The effect of the third booster vaccination on the incidence of COVID-19 was estimated in various subgroups to investigate the clinical factors affecting prophylactic efficacy (figure 3). Patient’s age was the most important factor (p value for interaction=0.036); the booster vaccination significantly decreased the incidence of COVID-19 in the subgroup of patients aged <70 years (HR 0.81 (95% CI 0.69 to 0.95)), whereas it did not decrease the incidence in older patients (HR 1.45 (95% CI 0.84 to 2.52)). In contrast, in the non-AIRD group, the effect of the third booster vaccine was more prominent in older patients (HR 0.42 (95% CI 0.17 to 1.01)) than in younger patients (HR 0.85 (95% CI 0.54 to 2.58)). The use of antirheumatic agents and presence of comorbidities did not have any significant interaction with vaccination outcomes.
Effect of the third booster vaccination stratified by various clinical factors.
Sensitivity analysis
We performed several sensitivity analyses to robust our results. First, COVID-19 infection rate was still comparable between AIRD and non-AIRD group after including 148 patients who were excluded because of short follow-up duration at the index date. (46.6% vs 44.4%, p=0.537)
Second, we evaluated risk factors for COVID-19 infection in patients with AIRDs by applying mutually exclusive categories to diagnosis and medications. The results of multivariable analysis were consistent to the original analysis. There was no significant association between AIRDs or antirheumatic medications between COVID-19 infection. Old age, GC use and booster vaccination showed lower COVID-19 infection (online supplemental table 5).
Third, the effects of the third booster vaccination did not change in the sensitivity analysis, in which the patients’ observation periods were further divided based on subsequent vaccination after 1 January 2022. From this analysis, booster vaccination decreased the incidence of COVID-19 only in patients younger than 70 years (adjusted HR 0.84 (95% CI 0.70 to 1.01)). In addition, since patients who received the third booster vaccinations were likely to have received their last vaccination more recently than those who received fewer vaccinations, the lower incidence of COVID-19 in these patients may be attributable to a more recent vaccination from the index date. Therefore, we assessed the effect of the third booster vaccinations in the multivariable model, including the interval between the last COVID-19 vaccination and the index date as a covariate. The result of this sensitivity analysis was also consistent with that in the original analysis (adjusted HR 0.80 (95% CI 0.66 to 0.96)).
Finally, because our study was an outpatient-based study, it is possible that patients with severe COVID-19 were more likely to be not included in this study due to missed visit and thus lead to selection bias. To estimate the potential bias, we assessed the all-cause mortality of patients who missed the scheduled visit during the survey period using MOSPA database. Although we did not assess the infection rate of COVID-19 in these patients, all-cause mortality rate during the observation period was only 3.7% (n=19). This result suggests that the effect of potential selection bias could be insignificant.
Discussion
Since December 2021, the Omicron variant has rapidly spread and become a dominant SARS-COV2 variant worldwide. During this era, South Korea has exhibited an unparalleled incidence of COVID-19, with more than 50% of the general population infected by 2022. The unique epidemiological situation in South Korea provided an optimal opportunity to investigate the impact of the crisis on patients with AIRDs who are known to be susceptible to COVID-19 and its complications. To the best of our knowledge, this is the first study to investigate the epidemiological features and clinical impact of the Omicron variant of COVID-19 in patients with AIRDs.
In this study, 46.0% of patients with AIRDs were infected with COVID-19 during the study period, with the rate being comparable to that in patients without AIRDs. The severity of the infection was milder than that of the other variants of COVID-19, and the hospitalisation rate was only 3.6%. This result is contrary to that of previous studies performed before the Omicron era, in which patients with AIRDs had an increased risk of COVID-19 and its complications.3 8 10–12 Our study also suggested that evasion of host immunity by Omicron is very effective, irrespective of the presence of underlying AIRDs or antirheumatic medications except GCs, which paradoxically showed lower infection. Since Omicron variant has a high capacity of immune evasion and relatively low virulence, the possibility of exposure and infection to COVID-19 could be higher with increased social contact. However, because previous studies before the Omicron era suggested that GC use was associated with increased risk of COVID-19 and related hospitalisation, the healthcare service in South Korea strongly recommended the limited social contact in the high-risk patients such as those receiving immunosuppressive agents. We think that that social circumstances in South Korea might lead to paradoxically showed lower infection rate of COVID-19 infection in patients with GC treatment.
Considering the high rate of booster vaccinations in the Korean population, the high incidence of the Omicron variant of COVID-19 is worth of note. In South Korea, the third booster vaccination began in October 2021, and more than 50% of the study population had received it at the index date. Emerging data from large population studies have shown that the Omicron variant evades neutralising antibodies in people vaccinated with one or two doses of the vaccine, and that the third booster vaccination is partially protective against the infection.20–23In addition, a recent cohort study performed in South Korea suggested that the mean cross-neutralising response against Omicron in patients with autoimmune diseases receiving the third dose of the mRNA vaccine was only 26.8%, which was significantly lower than that in a control healthcare worker group.24 However, few studies have investigated the effectiveness of booster vaccinations against the Omicron variant in a large number of patients with AIRDs.
In this study, we demonstrated that the third booster vaccination significantly but modestly decreased the risk of COVID-19 infection of the Omicron variant and that this protective effect in the AIRD group was comparable to that in the non-AIRD group. Furthermore, the effect of the booster vaccination was not impaired by comorbidities or immunosuppressive treatment. This suggests that a third booster vaccination should be universally considered in patients with AIRDs to reduce the risk of future COVID-19. This is also supported by the fact that the first and second doses of vaccination did not reduce the incidence of COVID-19 in this study.
Notably, patients with AIRDs aged >70 years were significantly associated with a lower risk of COVID-19 infection and did not significantly benefit from the third booster vaccination. In contrast, in the non-AIRD group, older patients showed lower COVID-19 incidence with booster vaccination. This can be explained by several factors. Because older patients with AIRDs were known to show high morbidity and mortality due to COVID-19 infection before the Omicron era, it was likely that they self-restricted social contact and therefore had a lower chance of exposure to COVID-19 than younger patients. In addition, previous studies have suggested that older people showed rapid waning of vaccine effectiveness. Because patients with AIRDs also showed decreased persistence of immunity against COVID-19 after vaccination, old age and AIRDs could have a synergistic effect on vaccine response. Therefore, the reduced efficacy of booster vaccination in older patients with AIRDs and their serological response should be further investigated to determine the optimal preventive methods in this group.
This study had some limitations. First, data regarding the confirmation of infection and its date were collected based on self-reported questionnaires, which could have led to recall bias. However, to minimise potential bias, we also checked the SMS messages for the diagnosis of COVID-19 infection sent from the KDCA. However, despite our effort to minimise the bias, it is still possible that infection rate of COVID-19 could be underestimated due to unmeasured factors such as undetected, asymptomatic infection. Second, although our cohort included more than 1500 patients with AIRDs, the number of patients with some individual diseases (eg, anti-neutrophil cytoplasmic antibody (ANCA) associated vasculitis) was rather small. Therefore, the effect of individual diseases and their treatment on COVID-19 should be further investigated in a larger-scale study. Finally, as this study included only patients who visited the clinic during the study period, the vaccine efficacy and clinical outcome of the Omicron variant of COVID-19 may have been overestimated due to the selection of patients with better outcomes. However, among the 515 patients lost to follow-up during the study period, only 19 died, suggesting that this potential bias was insignificant.
In conclusion, we demonstrated that the incidence rate and outcome of Omicron variant infections in patients with AIRDs are comparable to those in patients without AIRDs. Furthermore, our findings indicate that the third booster vaccination significantly reduces the incidence of COVID-19 in patients with AIRDs, particularly in those aged <70 years. Although further studies are necessary to confirm our results, our findings may be crucial to develop strategies against COVID-19 in patients with AIRDs.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was conducted in accordance with tenets of the Declaration of Helsinki and approved by the Institutional Review Board of Seoul National University Hospital (IRB No. 2207-027-1337). Written informed consent was obtained from all patients.
Acknowledgments
We thank all the study participants for their support of the study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors YKK and JWP conceived and designed the study. Recruitment and screening of study subjects was performed by JKP, EYL, E-BL and JWP. Data were collected and interpreted by YKK, JIJ, JKP, EYL and E-BL. Statistical analysis was performed by YKK, JYK and JWP. The manuscript was written by YKK and JWP. All authors contributed to the critical revision of the manuscript and approved the final manuscript. JWP had access to data, accepted full responsibility for the work and/or the conduct of the study, controlled the decision to publish and acted as guarantors for the study.
Funding This research work was supported by Seoul National University Hospital Research Fund (grant number: 0420220760). Also, this work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (RS-2023-00251803)
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.