Article Text
PDF
Abstract
Objective Evaluate the performance of FRAX®, with and without bone mineral densitometry (BMD), in predicting the occurrence of fragility fractures over 10 years.
Methods Participants aged ≥40 years at baseline, with a complete set of data and a minimum of 8.5 years of follow-up were identified from three cohorts (n=2626). Ten-year fracture risk at baseline were estimated with FRAX® and assessed by comparison with observed fractures and receiver operating characteristic analysis.
Results During a mean (SD) follow-up of 9.12 (1.5) years, 178 participants suffered a major osteoporotic (MOP) fracture and 28 sustained a hip fracture. The predictive performance of FRAX® was superior to that of BMD alone for both MOP and hip fractures. The area under the curve (AUC) of FRAX® without BMD was 0.76 (95% CI 0.72 to 0.79) for MOP fractures and 0.78 (95% CI 0.69 to 0.86) for hip fractures. No significant improvements were found when BMD was added to clinical variables to predict either MOP (0.78, 95% CI 0.74 to 0.82, p=0.25) or hip fractures (0.79, 95% CI 0.69 to 0.89, p=0.72).
AUCs for FRAX® (with and without BMD) were greater for men than for women. FRAX®, with and without BMD, tended to underestimate the number of MOP fractures and to overestimate the number of hip fractures in females. In men, the number of observed fractures were within the 95% CI of the number predicted, both with and without BMD.
Conclusion FRAX® without BMD provided good fracture prediction. Adding BMD to FRAX® did not improve the performance of the tool in the general population.
- osteoporosis
- epidemiology
- outcomes research
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key messages
What is already know about this subject?
FRAX® has been validated and is used in a large number of countries to estimate the risk of osteoporotic fractures, thus informing individual treatment, societal preventive strategies and national guidelines to initiate treatment. However, a recent systematic literature review identified a large number of important limitations and caveats in the available studies.
What does this study add?
This study provides a methodologically robust piece of evidence supporting the predictive value of FRAX® (area under the curve: 0.72 to 0.93) in a general population setting. It also demonstrates, for the first time that, at a global level, the performance of this tool is not enhanced by considering bone mineral density in addition to immediately available clinical risk factors.
How might this impact on clinical practice?
The data presented on this study substantially adds, in quantity and quality, to the evidence supporting the use of FRAX® to estimate the risk of fracture and serve as a reference in the decision to treat. The data indicates that, overall, estimates based solely on the clinical risk factors included in the algorythmhave a similar reliability, questionning the need for systematic assessment of bone mineral density.
Introduction
Osteoporotic fractures currently represent an enormous social and economic burden worldwide,1 which will tend to increase persistently due to the progressive ageing of the population and other societal changes,2 unless effective preventive measures are taken.
For cost-effectiveness purposes, preventive strategies should be based on the absolute risk of osteoporotic fractures in the individual patient. FRAX®3 4 is the most widely used tool to estimate osteoporotic fracture probabilities,5 and it has been incorporated in a large number of guidelines for the prevention and management of osteoporosis.2 6–9 FRAX® estimates are based on a set of easily assessable clinical risk factors, with or without consideration of femoral neck bone mineral density (BMD),4 making it a feasible tool, even in technically deprived environments.
Given the differences in the incidence of major osteoporotic (MOP) fractures between countries,10 11 FRAX® should be validated in national cohorts to optimise its predictive value in each country.4 A recent systematic review12 demonstrated that this has not always been done and that most validation studies have significant bias, especially recruitment bias regarding the target population, and missing data on clinical risk factors. Few of these studies worldwide have been conducted in the general population.12
The purpose of this study was to evaluate the performance of FRAX® in predicting the 10-year probability of osteoporotic fractures using data from three prospective cohorts from the general population. We also investigated the value of adding BMD to the clinical parameters of FRAX®.
Methods
For this study, data of three different Portuguese cohorts, SAOL, IPR and EPIPorto (from centre, south and north of the country, respectively), were combined. Only persons aged >40 years and with a complete set of data on FRAX® clinical risk factors were included. There were no other exclusion criteria.
More details on cohorts and selection of participants can be found on online supplementary material. Online supplementary figure 1 shows the disposition of participants during follow-up and numbers used for data analysis.
Supplementary file 1
BMD evaluation
Dual energy X-ray absorptiometry (DXA) scans of the spine and proximal femur of the non-dominant side were performed at the baseline visit of all participants, using a Hologic QDR 4500/c bone densitometer in all cases. A daily quality exam was performed every day to ensure the quality of the exams performed in the bone densitometer (as recommended by the manufacture); a well-trained technician in every cohort performed the exams. Participants without femoral BMD measurement at baseline were excluded. Hip T scores were used as provided by the bone densitometer on the basis of NHANES III (Third National Health and Nutrition Examination Survey) reference values.13
Fractures
The first new fracture during follow-up and the date on which it occurred were self-reported at the follow-up visit in all cohorts. In the SAOL cohort, fracture reports were confirmed by clinical file review in all but 2 of 52 fractures.
The fracture outcome of interest in this analysis was new first hip fracture and fracture of either the hip, wrist, shoulder or clinical fracture of the spine (MOP), regardless of the degree of trauma, so as to conform to the definition of hip, and MOP fracture by FRAX®.
FRAX® predictions
The 10-year fracture risk estimates for hip and MOP fractures (with and without adding the variable femoral neck BMD) for each individual case were assessed using the Portuguese version of the FRAX® tool by an operator who was blinded for the fracture outcomes. All variables were defined exactly as prescribed by FRAX®.
All three cohort studies had been approved by local ethics committees, and informed consent had been obtained from all patients. The Research Ethics Board of Faculty of Medicine of Coimbra University approved the current analysis.
Statistics
Follow-up time for the fracture analyses was truncated at 10 years, when applicable, to correspond with the 10-year fracture risk estimates from FRAX®. Of participants who deceased during follow-up (n=292), fracture data were collected from family members and included in the analyses, according to the assumption of the tool.10 Data for survival analyses was censored at the date of first fracture, date of death or 10 years without fractures or end of follow-up before 10 years without fractures (as described in Methods, participants from EPIPorto did not complete 10 years of follow-up).
Descriptive statistics for demographic and baseline characteristics are presented as mean (SD) or median (IQR) for continuous variables or count (percentage) for categorical variables.
Crude comparisons of parameters of participants with fractures versus those with no fractures were performed with χ2 tests and Mann-Whitney U test.
Cox proportional hazards models were constructed for MOP and for hip fracture prediction separately, to assess the contributions of the individual FRAX® variables. Cox proportional hazards takes time into account, thus the shorter duration of follow-up in EPIPorto was not an issue for those who had a first new fracture during this follow-up. Receiver operating characteristic (ROC) area under the curve (AUC) analyses were conducted to explore the fracture risk stratification using FRAX® with and without BMD and the prediction of BMD alone (femoral neck T score or minimum value at any site). An AUC of 0.50 indicates a result no better than chance, an AUC >0.5–0.6 or <0.5–0.4: poor discriminative value, 0.6–0.8 or 0.2–0.4: moderate discriminative value and >0.8 or <0.2: high discriminative value14; only AUCs with CI excluding 0.5 are statistically significant. Pairwise comparison of AUCs ROC was performed using MedCalc (V.14.8.1). Sensitivity analyses were performed by excluding data from EPIPorto given their shorter follow-up. Kaplan-Meier curves were plotted, showing fracture incidence over time by cohort.
We assessed the fit of predicted values of FRAX® by comparing the observed proportion of participants who sustained a first new fracture with the proportion predicted by FRAX®. These analyses were undertaken in the entire cohort and then repeated in the cohort divided into clinically relevant subgroups for age and gender.
Statistical analyses were performed with SPSS for Windows (V.20.0). We applied STROBE (Strengthening the reporting of oservational studies in epidemiology) criteria for cohort studies to ensure the quality of our study.15 A p value of <0.05 was taken as statistically significant.
Results
The study sample with baseline and follow-up observations consisted of 2626 participants (1943 women (73%) and 683 (27%) men). Baseline characteristics are summarised in table 1. The mean (SD) age at baseline was 58.2 (10.2) years; during follow-up, 292 (11.1%) participants had died from different causes. The most prevalent among FRAX® clinical risk factors was ‘secondary osteoporosis’ (24.3%), and the least prevalent was rheumatoid arthritis (4.9%).
Baseline characteristics of participants and baseline FRAX® risk estimates
During follow-up, with a mean (SD) duration of 9.12 (1.5) years (minimum and a total 23 949 person/years), 28 (1.1%) of these participants suffered from an incident hip fracture (median FRAX®-estimated risk at baseline for hip fracture: without BMD 2.8% (1.4–4.8); with BMD 6.9% (1.9–11.8)) and 178 (6.8%) had an incident MOP (median FRAX®-estimated risk at baseline for MOP: without BMD 6.7% (3.9–10); with BMD 8.9% (5.2–14)). More details can be found in online supplementary table 1.
In table 2, we present the ROC AUC for FRAX® estimates, with and without BMD, as well as ROC AUC for DXA alone. The performance of FRAX® is superior to that for DXA alone, for both MOP and hip fractures, in both men and women. Please see also online supplementary figures 2 and 3.
ROC area under the curve (AUC) analyses for hip and major osteoporotic fractures
AUCs based on FRAX® were numerically higher when including DXA, compared with not including DXA, with the exception of hip fractures in men, but none of these differences reached statistical significance. AUCs based on FRAX® with DXA were higher to those of DXA alone, all differences being statistically significant. ROC analyses excluding participants from EPIPorto revealed exactly the same AUC values, except for a modest increase for hip prediction with BMD (AUC 0.80, 95% CI 0.71 to 0.89) (data not shown).
As shown in table 3, when BMD was not included in the model, all clinical risk factors except BMI and rheumatoid arthritis were independent predictors of new MOP fractures. Regarding new hip fractures, the model without BMD retains age and glucocorticoids as significant predictors associated with a history of parent hip fractures.
HRs for fracture based on individual FRAX® variables excluding BMD. All variables are defined as prescribed by FRAX®
When BMD was included in the model (table 4), age, glucocorticoids, parent hip fractures, previous osteoporotic fracture, current smoking, secondary osteoporosis and femoral neck BMD were all independent predictors of MOP fractures in our sample. In the model with BMD, parental hip fractures showed the largest predicted risk for MOP fracture (HR 3.69, 95% CI 2.51 to 5.43) and BMD showed the smallest (HR 0.72, 95% CI 0.62 to 0.83).
HRs for fracture based on individual FRAX® variables including femoral neck BMD
The only independent predictors of hip fractures were age, BMI and femoral neck BMD. Gender, alcohol usage, secondary osteoporosis and rheumatoid arthritis were not independently associated with either MOP or hip fractures.
Table 5 shows the calibration of each calculator by comparing the number of observed first new fractures and the at baseline estimated risk by FRAX® (95% CI). FRAX® with and without BMD underestimated incident MOP and overestimated hip fractures in women. In men, the observed rates of first new fractures were within the 95% CI of baseline FRAX® predicted rates.
FRAX®-estimated and observed number of first new fractures during follow-up
Regarding age and considering both genders together, the observed number of hip fractures was within the 95% CI of prediction in all ages groups, with the exception of underestimation of the FRAX® with BMD estimate for the group aged <60 years. For MOP fractures, the baseline FRAX® estimate also underestimated the risk for those under the age of 75 years. The agreement between predicted and observed rats above this age was better, although the number of observed new first fractures was small.
Kaplan-Meier survival models showed that a similar number of new first fractures occurs every year in all three cohorts (data not shown). Based on this observation, we estimated that 12 MOP fractures would have occurred in the 2.5 missing years of follow-up in EPIPorto, which are unaccounted for. The impact of this difference of follow-up was evaluated through sensitivity analysis, as described above.
Discussion
In this study in a cohort of general population, we used several approaches to compare the predictive performance of baseline FRAX® estimates with and without BMD with observed first new fractures during follow-up. FRAX® is designed to predict the risk of fracture and does not distinguish the risk of multiple fractures in one individual. That is why our data were censored at the first fracture.
AUC ROC values of baseline FRAX® estimates ranged from 0.78 to 0.79 for hip fracture and from 0.76 to 0.78 for MOP fracture, indicating moderate discriminative ability of FRAX®, with and without BMD, for predicting both hip and MOP fractures in both genders. The AUC ROC values for BMD alone were 0.72 for hip fractures and 0.69 for MOP fracture. FRAX® estimates with and without BMD have a better performance than has BMD alone. No significant differences were found between the predictive performance of FRAX® estimates with and without BMD.
The prediction of first new hip fractures was more reliable than that of MOP fractures, which is in agreement with previous studies.16–22 The performance of the tool was higher in males than in females, for both groups of fractures.18 23
AUC ROC values found in our study are generally higher than those found in a recent meta-analysis.12 This may be related to the higher quality of methods used in our study and the full respect for the conditions of FRAX® applicability predicted in its development process4 10; in contrast with most previous studies, we included participants from the general population and considered all clinical risk factors included in FRAX® using the exact definitions provided by the tool; we only included cases with a complete set of clinical data, and we accounted for participants who died during follow-up. Our only limitation in this respect was the shorter duration of follow-up in one of the cohorts. The prevalence of clinical risk factors was similar to other studies, with exception of secondary osteoporosis and consumption of alcohol and tobacco that were higher in our study, which may be related to the systematic questioning about these risk factors. We also found that the individual risk factors used by FRAX® had significant independent contributions to fracture prediction. Age and glucocorticoid use were strongly associated with new first MOP and hip fracture risk in both models (with and without BMD).
FRAX®, with and without BMD, underestimated incident MOP and overestimated hip fractures in women, while in men the observed number of both types of fractures was within the 95% CI of the prediction, both with and without BMD. These results are similar with those found in other studies16 20 21 24 25 and a systematic review.26 We hypothesise that the discrepancies between the observed and predicted rates of fractures in females may be related to the fact that for the construction of Portuguese FRAX® algorithm, we have only used actual national epidemiological data for hip fractures, the rate for the MOP fractures being estimated using Swedish age specific ratios.27 When considering age, the number of observed first new MOP fractures was higher than estimated until the age of 75 years. There was good agreement regarding hip fractures in all age groups.
Overall, our results show that the FRAX® algorithm with clinical risk factors has a better performance at predicting the rate of first new fractures than BMD alone. This is in agreement with previous studies.3 18 They also demonstrate that adding BMD to the clinical risk factors brings no improvement to FRAX® prediction, in terms of AUC ROC or rates of observed versus predicted fractures. This is the case in both men and women, MOP and hip fractures. The impact of DXA on FRAX® performance has, to the best of our knowledge, not been investigated before. Our observations question the cost-effectiveness of DXA measurements for the purpose of predicting fractures in the general population.
Some limitations of this study need to be acknowledged. Our dropout rate was 25.2%, which is considerable, although similar to that reported in other prospective cohort studies of FRAX®.28–30 Patients lost to follow-up did not differ significantly for those included in the analyses in terms of clinical risk factors included in FRAX® (data not shown). Fracture events were self-reported and only confirmed in the SAOL cohort by clinical file review. During follow-up, 7.6% of participants used antiosteoporotic agents at some time, which reflects the low rate of treatment of osteoporosis in the general Portuguese population.31 Such treatment may have prevented some fractures, potentially contributing to the overestimation of the fracture risk, even though recent studies have been unable to show this effect in open population-based studies.32 33 We investigated the performance of FRAX® Portugal,we are unable to comment on the calibration or discrimination of other country-specific FRAX® tools. Follow-up in the EPIPorto cohort was shorter than the 10-year timeline of FRAX®. This did not have a relevant impact regarding the ROC analyses but may have artificially reduced, although slightly, the underestimation of the actual number of fractures by FRAX®.
Our study has several strengths: it is a multicentre cohort of participants recruited from the general population, the average duration of follow-up was 8.7 years, the clinical risk factors included in FRAX® were collected in all participants and we also considered death hazard. These qualities support the validity of the results in the population expected to receive the test in daily practice.
The FRAX® tool has been incorporated in the clinical guidelines for osteoporosis of several countries.34 The data presented in this paper have already influenced the recent Portuguese recommendations35; both the requirement of DEXA (Dual-energy X-ray absorptiometry) and the initiation of treatment are based on estimates of the actual risk of fracture by FRAX®, with or without BMD T scores
Conclusion
A moderate performance of FRAX® was found in both men and women, with higher AUCs ROC than those reported in the derivation and validation cohorts studied by the WHO Collaborating Centre3 and considered in a recent meta-analyses.12 The performance was better for hip than for MOP fractures, in males than in females, and in participants over the age of 75 years. Adding BMD to the model did not improve FRAX® performance.
References
Footnotes
Contributors AM and JAPS were responsible for conceiving the project. AM and RL were responsible for data collection. AM performed the data management and statistical analysis in collaboration with RL, SMMV, JJ and JAPS. AM wrote the paper in collaboration with all coauthors. All coauthors read the report and made suggestions about its content.
Funding This study was supported by unrestricted grants from the Direção Geral da Saúde and Amgen, which had no role in the design of the study, the writing or review of the paper.
Competing interests None declared.
Patient consent Obtained.
Ethics approval The Research Ethics Board of Faculty of Medicine of Coimbra University approved the current analysis. All participants gave informed consent before taking part.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.