Abstract
Objective To investigate the performance of a health app with respect to usability, adherence, and equivalence of data in daily care of patients with axial spondyloarthritis (axSpA).
Methods Consecutive patients with axSpA were asked to export patient-reported outcomes (PRO) electronically with the AxSpA Live App regularly every 2 weeks over a period of 6 months. The first clinical visit was followed by 2 further personal visits after 3 and 6 months. Patients completed paper-based PRO at every visit; they also completed the Mobile App Rating Scale and the System Usability Scale after 3 and 6 months.
Results Of 103 patients with axSpA, 69 agreed to participate (67.0%): age 41.5 (11.3) years, 58.0% male, Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) 4.3 (2.0), and 76.8% treated with biologic disease-modifying antirheumatic drugs. Patients’ adherence to regular app exports was 29.0% and 28.4% after 3 and 6 months, respectively. Significant predictors for good adherence were high disease activity (P = 0.02) and older age (P = 0.04). No systematic differences between digital and paper-based BASDAI scores were found (intraclass correlation coefficients 0.99 [95% CI 0.98-0.99]). Performance of the app was rated as good.
Conclusion Collection of digital PROs by AxSpA Live App may be successfully used in patients with axSpA with high disease activity. Our study showed equivalence of digital data, but adherence to the app after 6 months was poor. Higher disease activity and older age resulted in increased adherence to the app. This suggests that the use of health apps like this should concentrate on more severely affected patients.
Axial spondyloarthritis (axSpA) is an inflammatory rheumatic disease associated with variable symptoms.1 Both medical and nonmedical treatment strategies based on the treat-to-target (T2T) principle are recommended,2,3 and tight monitoring with regular assessments has been defined as a quality standard.4 However, time limitations of healthcare professionals, leading to a shortage of rheumatologic appointments, warrant new strategies for care of patients with axSpA. Application of electronic devices may improve the disease management of patients with axSpA.
Regular assessment of patients’ health status currently focuses on disease activity and partly on physical function.5 This is especially relevant for patients with high disease activity who started treatment with nonsteroidal antiinflammatory drugs (NSAIDs).5 According to the Assessment of SpondyloArthritis international Society (ASAS) recommendations, at least 2 different NSAIDs should be given over a period of 4 weeks, which implicitly means that disease activity must be checked every 2 weeks. Patient-reported outcomes (PROs) are usually assessed with paper-based questionnaires for disease activity, with either the Bath Ankylosing Spondylitis (AS) Disease Activity Index (BASDAI) or the AS Disease Activity Score (ASDAS).6,7 Although these scores are well implemented in clinical routine,5 the use of paper-based questionnaires lacks flexibility.8 Further, the current COVID-19 (coronavirus disease 2019) pandemic has partly made personal visits problematic, highlighting the need for alternative ways of communication with patients.9
Digital solutions such as electronic PRO (ePRO) offer great potential for implementing patient-centered communication and better executing adaptive T2T strategies.8 Equivalence between digital and paper-based PROs has already been shown in previous studies.10,11 Health apps provide opportunities to improve communication between patients and physicians and empower patients to manage their disease independently.12,13 There is evidence that closer monitoring of patients with rheumatoid arthritis (RA) has a positive effect on clinical outcomes.14 Since the majority of assessments used in axSpA are patient-reported, health apps may help to save resources and facilitate data acquisition to improve efficiency in clinical routine.8
To our knowledge, no such studies have been carried out in patients with axSpA. The aim of this project was to study the use of a health app with respect to usability, adherence, and equivalence of data in the daily care of patients with axSpA.
METHODS
Study population. This single-center longitudinal study was performed in our tertiary rheumatology center between November 2019 and July 2020. Consecutive adult patients diagnosed with axSpA who also fulfilled ASAS classification criteria were prospectively included after having documented informed consent.15 Patients were excluded if they did not possess a smartphone, did not speak German, or were unable to use the AxSpA Live App. The study was approved by the Ethical Committee of the Ruhr-Universität Bochum, Germany (Reg.-Nr.:18-66841).
Health app. Patients used the German AxSpA Live App, a certified medical app classified to the lowest risk level (Class 1 medical device) of the Medical Device Regulation of the European Union. Patients were asked to document disease activity by BASDAI and pain level (numerical rating scale score 0-10, 10 indicating most severe pain) every 2 weeks. The time frames chosen were based on the ASAS recommendation to monitor disease activity every 2 weeks in patients with high disease activity starting medication with NSAIDs. The AxSpA Live App was available free of charge for both Android and iOS systems and was distributed by STAR Healthcare Management GmbH (Germany). The AxSpA Live App offers a diary function in which various disease variables (eg, disease activity, physical function, pain) can be documented daily. In the diary, patients can view their most recent entries and have them graphically displayed in a curve. The AxSpA Live App contains an internal reminder function, which was preinstalled for a time window of 4 weeks in 50% of the study population. The reminder was installed to investigate whether its activation had an influence on export behavior.
The ePRO results can be exported to the treating rheumatologist by generating a secure patient/physician interface through the use of an individual QR code to connect the patient module with the module of the treating physician. Incoming ePRO results were checked by the study team and results were documented in the hospital information system.
Study design. Patients were seen face-to-face at baseline (V1) and weeks 12 (V2) and 24 (V3) during a planned routine appointment in our outpatient clinic (Figure 1). For V2 and V3, an extended time window of ± 4 weeks was tolerated. Thus, the total duration of the study was 24 weeks plus 4 weeks for possible deviations. In order for patients to use the app properly, they received standardized training at baseline about the functionality of the AxSpA Live App. Installation of the AxSpA Live App on the personal smartphones of the patient and connection between patient and physician module took place in the hospital to guarantee functioning of the app. The start of export was defined as the week 2 visit after the initial face-to-face visit. For data export, an extended time window of ± 7 days was tolerated. Export of data at the face-to-face visits was necessary to test equivalence but did not count among the time windows in which data were supposed to be exported. Thus, there were 5 timepoints planned within a 12-week period in which the patient was asked to export ePRO data. Adherence to ePRO export was defined as an export of BASDAI scores on at least 4 of the 5 proposed timepoints. All patients who had not exported any data within the first 4 weeks were reminded by telephone to check potential technical problems and then to resolve them. All patients were again reminded on the face-to-face visits at V2 and V3.
Clinical data. At baseline, patients and disease characteristics were documented, and patients provided information on their previous experience with electronic media (5-point Likert scale from very rare to very frequent) and whether they use other health apps. If patients used electronic media several times a day, the patient was defined as a frequent user. At baseline, V2, and V3, patients completed paper-based BASDAI and pain scores. BASDAI and pain scores were taken at V2 digitally and on paper to allow test of equivalence. Patients were informed that they should respond to the questionnaires twice. The ASDAS based on C-reactive protein (CRP) was also collected at all face-to-face visits.7 ASDAS thresholds were used to categorize disease activity states.16 NSAID intake was assessed by the ASAS-NSAID score (range 0 [no intake] to 100 [maximum intake]).17 Medications at baseline were documented with a special emphasis on start of new NSAIDs and/or bDMARDs.
Quality and usability of the AxSpA Live App were examined by the Mobile App Rating Scale (MARS) and the System Usability Scale (SUS) at V2 and V3.18,19 MARS is a multidimensional rating scale for mobile applications based on a 5-point Likert scale that consists of 4 sections: (A) Engagement, (B) Functionality, (C) Aesthetics, and (D) Information (sum score 1 and 5, with higher scores indicating better app quality). An additional section assesses the subjective quality of the application. Here, users can indicate, for instance, whether they would recommend the app to someone else or would purchase it if it were fee-based. In addition, users could rate whether the app had a positive effect on their health behavior. The evaluation thus results in an average value for each section and an average value for the overall app quality between 1 to 5. A value of ≥ 3 was considered moderate and a value of ≥ 4 was considered good.20 SUS is a 10-item questionnaire to classify the usability of an application (sum score 0 and 100, with higher scores indicating higher usability). A value of ≥ 62 was considered acceptable, and a value ≥ 68 as above-average usability.21 At V3, patients were additionally asked whether they would continue to use the AxSpA Live App after the study (yes/no/maybe).
Statistical methods. Study results presented as mean (SD) or median for quantitative variables, and as absolute and percentage frequencies for qualitative variables. Intraclass correlation coefficients (ICC) with 95% CIs were used to test equivalence between paper-based and electronic questionnaires. An ICC of ≥ 0.80 was considered as excellent equivalence.22 Mann-Whitney U test was used for metric scale variables and the chi-square test for categorical variables. P < 0.05 was considered to indicate statistical significance for both tests.
RESULTS
Study population. Of 103 patients approached, 69 patients (67%) agreed to participate in the study and were recruited between November 2019 and January 2020. Five patients did not have a smartphone, 1 was unable to download the app for technical reasons, and 28 reported other mostly personal reasons. The 69 patients included had a mean age of 41.5 (SD 11.3) years, and 58% were male (Table 1). Disease activity at baseline assessed by paper-based form was relatively high, with a mean (SD) BASDAI of 4.3 (2.0) and ASDAS of 2.5 (1.0; Table 2). Patients who declined study participation did not differ in their characteristics from patients who agreed to participate; the exception was for patients starting a new treatment, who were underrepresented in the nonparticipant group (Table 3). Patients used electronic media frequently, with 62 patients (89.9%) reporting frequent or very frequent electronic media usage. Thirty-five (50.7%) of the patients had used health apps (eg, pedometer, fitness app) in daily life before.
Follow-up period. Of the 69 patients included, 67 participated in V2 (97.1%) and 64 in V3 (92.8%). Because of the COVID-19 pandemic, 10 patients in V2 and 7 patients in V3 did not have a face-to-face visit between March and July 2020. The visit was performed by telephone. In this consultation, all study-specific variables were collected. Mean disease activity and CRP levels remained stable during the 24-week study period (Table 2). There was a slight increase in the ASAS-NSAID score until week 24. During the first 2 weeks AxSpA Live App data (BASDAI or pain level) were exported by 45 patients (65.2%). After 4 weeks, a total of 56 patients (81.1%) had exported health data at least once. Thirteen patients (18.8%) were reminded at week 4 by telephone to export data because of missing transmission. Only 5 patients (7.2%) did not send any data at all within the first 12 weeks. The number of patients who did not use the AxSpA Live App increased after week 12. After V2, 14 patients (20.1%) did not export any data until the end of the study. The internal reminder function of the app had no significant influence on whether patients exported their health data. After 4 weeks, more data had been sent in by patients who had not been reminded than from patients who had been reminded.
The mean number of BASDAI exports by patients between V1 and V2 was 2.5 (1.6), with a median of 3.0. After completion of V2, this value declined to 2.1 (1.8), with a median of 2.0. These values and Figure 2 show that there was an overall decrease in patients’ export behavior after V2 (Figure 2). Patients with high disease activity exported BASDAI values more frequently compared to patients with low ASDAS scores < 2.1. The proportion of patients with high disease activity who exported the BASDAI scores electronically on a regular basis decreased from 40% to 30% between weeks 12 and 24. The proportion of patients with low disease activity who exported BASDAI scores electronically on a regular basis increased from 12% to 24% between weeks 12 and 24.
At week 12, only 20 patients (29.0%) were found to be app-adherent, meaning that they had completed 4 of 5 exports. Of the 67 patients who were still participating in V2, 19 patients (28.4%) remained adherent in V3. Thus, the number of app-adherent patients remained approximately the same over the complete study period. Thirteen patients (18.8%) were app-adherent at both V2 and V3. App-adherent patients at V2 were significantly older (P = 0.04) and showed higher disease activity scores (P = 0.02; Table 1). Overall, 85% of app-adherent users at V2 had an ASDAS ≥ 2.1. Further, app-adherent users more often had an elevated CRP level and a higher mean NSAID score. In addition, app-adherent users were significantly less likely to have been treated with bDMARDs at the time of inclusion into the study. In contrast, sex, education level, frequent use of the smartphone, or use of other health apps had no significant influence on adherence to the app (Table 1).
Equivalence of data. The comparison between digital and paper-based assessments did not show a significant difference on V2. The ICC for pain levels at V2 was 0.95 (95% CI 0.91-0.97). The equivalence for BASDAI scores, tested at V2, showed an ICC of 0.99 (95% CI 0.98-0.99; data not shown).
Evaluation of the AxSpA Live App. The quality of the AxSpA Live App was evaluated as being good, with a mean (SD) MARS score of 3.6 (0.6; Figure 3). In the functionality category, the AxSpA Live App achieved the best ratings, with a score of 4.0 (0.6). In the engagement category, the app achieved the weakest ratings, with an average of 3.1 (0.8). The effect of the app on patients’ health behavior was rated as not strong, with a mean score of 2.9 (0.9). After 24 weeks, the quality of the AxSpA Live App was rated similarly. The usability of the AxSpA Live App as assessed by SUS achieved an above-average rating with a mean of 71.2 (17.7) and did not change much after 24 weeks (data not shown).
End of study evaluation. Of 63 patients surveyed at the end of the study, only 12 patients (19.0%) said they would continue to use the app, whereas 20 patients (31.7%) indicated that it was unlikely that they would continue to use the app. However, almost half of the patients (n = 31, 49.2%) stated that they might continue to use the app after the end of the study. The patients who indicated further use of the app rated usability significantly higher on MARS (P = 0.01) and SUS (P = 0.02) at V3 compared to those who indicated not to plan use of the app in the future. These patients were also significantly more likely to see a positive effect on their health behavior due to the app (P = 0.04) and were more likely to be app-adherent (P = 0.03) at V3 as compared to patients who did not want to continue to use the app (data not shown).
DISCUSSION
In this proof-of-concept study, an axSpA-specific health app was investigated for the first time longitudinally in patients with axSpA. In this representative cohort of patients with axSpA, a large proportion consented to use the AxSpA Live App regularly every 2 weeks. In fact, a relatively high number of patients did use the AxSpA Live App and exported ePRO data. Thus, the user rate was 65.2% after 2 weeks and as much as 81.1% after 4 weeks. In the first 12 weeks, only a few patients (7.2%) did not export any health data. However, less than one-third of patients used the app regularly every 2 weeks over 12 weeks and were therefore classified as app-adherent. This suggests that in the beginning, most patients with axSpA included in the study were motivated and willing to use the app, but many lost interest later and did not actively participate after a few weeks. However, a 2-week interval over a period of 24 weeks is very ambitious and may have contributed to the low adherence rate.
Patients with older age and high disease activity were more likely to use the AxSpA Live App. This is in line with our expectations that patients with higher disease activity use a health app more frequently because they have a higher burden of disease. Thus, they are more willing to explore different treatment concepts and “new” tools such as health apps. Especially for these patients, a health app may well improve patient care (eg, by inducing a medical appointment in case of worsening of their condition). The ability to export data and the usability of a health app are important prerequisites because staff do not have time to solve technical problems. This is especially relevant in times of low personnel resources as well as when adapting therapy and deciding on treatment if patients did not reach target. The diary function in the app could be used as a self-monitoring tool, but this was not explicitly monitored in our study.
Until now, comparable studies exist only for RA. A German study showed that the use of an app for disease monitoring was well accepted by patients and seemed feasible.23 The study showed an app retention rate of 71.7% at 3 months by investigating ePRO documentation intervals.23 Another recent study on the use of a health app with weekly entries to improve self-monitoring of patients with RA showed a significant decline of app users within 4 weeks.24 In a recent study on daily ePRO questionnaires, there was significantly higher adherence rates, with a median adherence to daily questionnaires of 79% over 6 months.25 Patients here, however, were reminded more frequently of app entries by telephone calls.25 These findings were supported by data of another study in which 20 patients were asked to document their health status by app daily for 3 months.26 The median exported data of 91% showed a high adherence rate in this intensive care study.26 It seems likely that more frequent (eg, daily or weekly) app entries lead to better compliance of patients, since they better integrate the use of the app into their daily routine. In addition, an intensified care of patients (eg, through regular reminder calls or additional physician contact) is likely to increase adherence. However, this approach requires more time and more personal resources, which makes its use not feasible for routine care in many centers.
In a Dutch study, an eHealth system for ePRO acquisition in patients with SpA was successfully implemented.27 The system was shown to improve communication between patients and care providers. Patients saw the eHealth system as an additional benefit to their health care. On the other hand, healthcare providers valued the additional information for their consultations. However, here, too, there were obstacles, such as the additional time required to digitize the data.
Our study demonstrates excellent agreement between the results of paper-based and app-based assessments for disease activity and pain scores. This is in line with previous studies showing equivalence of electronic and paper-based scores.10,28
Indeed, the quality and usability of the AxSpA Live App were rated as good to acceptable at both follow-up visits. At the end of the study, the number of patients who stated that they would continue to use the app was quite low at 19%. This subgroup not only rated the usability of the app significantly better and used it more regularly but also more frequently saw a positive influence of the app on their health behavior. Thus, a subjectively perceived improvement in health behavior seems to be an important factor for the constant use of a self-monitoring app such as the one used in our study. The number of patients willing to use a self-monitoring app may be increased if other benefits are offered more definitely, such as fewer physician visits when disease activity is low and replacing paper assessments with electronic assessments.8
In 2019, a large number of German-language rheumatology apps including the AxSpA Live App were evaluated by experts using MARS.29 The median MARS score for the AxSpA Live App in that publication was, with 3.6 points, identical to our study. Thus, the AxSpA Live App was rated as good to moderate by both experts and patients.
In another 2019 study, English self-monitoring apps for patients with axSpA were searched for, and then also assessed by MARS.30 The self-monitoring apps evaluated in this study showed MARS scores between 2.9 to 4.0.30 Thus, in an indirect comparison, the AxSpA Live App would rank in the middle range.
The present study has several limitations. The AxSpA Live App was tested only in a relatively small, although representative, cohort of patients with axSpA. Our study is monocentric, and only patients treated in our specialized tertiary center, the Rheumazentrum Ruhrgebiet, were included in the study. Therefore, it is possible that patients with a relatively more severe course and higher disease activity were included; this could have led to selection bias The definition of app adherence as 4 out of 5 exports was set rather strictly. This may have resulted in a relatively low adherence rate. The study duration of 6 months chosen for our study is not short but does not allow for conclusions about adherence rates over longer periods of time. Patients would possibly perceive a greater effect on their health status and care with the rheumatologist involved, and this could possibly increase app adherence. In this context, further studies are needed to evaluate whether and to what extent the treating physician finds the app and the generated exported data as helpful. Due to the COVID-19 pandemic, not all visits could take place in our outpatient clinic, which meant that telephone and postal visits had to replace personal visits in some cases. This may have negatively influenced adherence to the app in the longitudinal phase of the study. When analyzing app data, only those exported by the patient were considered. Data entered but not exported were not included. The time taken to install the app and complete the ePROs was not recorded.
In conclusion, collection of digital PROs by use of a health app may be successfully used in patients with axSpA with high disease activity in daily care. Our study showed equivalence of digital data but adherence to AxSpA Live App after 6 months was poor in the sample, probably due to the ambitious biweekly timeframe. Higher disease activity and older age resulted in increased adherence to the AxSpA Live App. Our intention is to study predictors of adherence in more detail to be able to identify those patients for which a health app will be most useful. As it stands now, it seems reasonable to concentrate on compliant patients with higher disease activity. The consequences for the time management of rheumatologists, the possibility of nurses educated to handle a health app, and economic consequences also deserve more study.
ACKNOWLEDGMENT
We thank the patients for their participation in the trial.
Footnotes
Parts of this study were funded by Novartis Pharma Germany.
The authors declare no conflicts of interest relevant to this article.
- Accepted for publication March 29, 2022.
- Copyright © 2022 by the Journal of Rheumatology