Multiple regression is perhaps one of the most widely used analytical procedures in the organizational sciences. Although many researchers use multiple regression to identify the set of variables that will maximize the prediction of some outcome variable or criterion, another important use of multiple regression is for theory testing and explanatory purposes. In these instances, researchers are most interested in understanding how each variable contributes toward the variance explained in the outcome or criterion. Alas, it is widely recognized (Darlington 1968) that multicollinearity makes the partitioning of variance among multiple correlated predictors difficult. Despite these difficulties, researchers continue to rely on commonly available indices (e.g., simple correlation coefficients, partial correlation coefficients, standardized regression coefficients) when partitioning the variance in the criterion that is predicted by multiple (typically correlated) predictors.

Fortunately, two alternative techniques, dominance analysis (Budescu 1993) and relative weight analysis (Johnson 2000), have been developed that permit more accurate partitioning of variance among correlated predictors. Simulation research clearly shows that these measures of importance perform quite well across a variety of conditions and are recommended for this purpose (LeBreton et al. 2004b). Despite calls advocating for the wider use of these indices (Tonidandel and LeBreton 2011), researchers seem reluctant to do so. One limiting factor for many researchers might lie in the practicalities associated with obtaining these metrics for their own datasets. The output typically generated by virtually all commercial statistical software fails to routinely include either of these indices in their standard output. Although some researchers have created macros and other similar variants that will perform these calculations, these resources rely on disparate statistical packages and fail to offer a simple, easy to use, integrated solution that would be attractive to most users. To that end, the purpose of this paper is to introduce a free, comprehensive, web-based, user-friendly resource to conduct relative weight analyses. In what follows, we will provide a brief overview of relative importance analyses, provide some examples from the published literature that makes use of these techniques, and introduce our comprehensive web-based solution for performing these calculations.

Relative Importance Analysis

Within the context of linear regression, the term relative importance refers to the proportionate contribution each predictor variable makes to the total predicted criterion variance taking into account a variable’s contribution by itself and in combination with other predictor variables (Johnson and LeBreton 2004). This definition of variable importance is not to be confused with other commonly used connotations of importance such as statistical or practical significance. Information regarding a variable’s contribution to predictable variance is helpful when considering the practical utility of a variable, but aspects of the particular situation must also be considered to fully gauge practical importance (Cortina and Landis 2009). In some situations, a variable may explain only a small proportion of predictable variance and yet be very meaningful (Martell et al. 1996), whereas in other situations, a variable may account for a larger percentage of the variance but may provide little practical utility (Cortina and Landis 2009).

Relative Weight Analysis

Relative weight analysis (RWA; Johnson 2000) addresses the problem caused by correlated predictors by using a variable transformation approach to create a set of new predictors (Z Xk) that are the maximally related to the original predictors (X j) but are orthogonal to one another. Because these new transformed predictors are uncorrelated to one another, the criterion variable (Y) can be regressed onto this new set of predictors producing a series of standardized regression coefficients (β k). Because these coefficients are generated using the orthogonal transformations of the original predictors, they no longer suffer from problems associated with collinearity. These regression coefficients are then rescaled back to the original variables by combining them with the standardized regression coefficients (λ jk) obtained by regressing the original predictors (X j) on their orthogonal counterparts (Z Xk) producing an estimate of relative importance for each predictor variable.

The interested reader is directed to Johnson (2000), Johnson and LeBreton (2004), LeBreton and Tonidandel (2008), and Tonidandel and LeBreton (2011) for a more detailed discussion of relative weight analysis. The relative weight statistic has been shown to provide extremely good estimates of the relative importance of predictor variables when those predictor variables are correlated. This has been found in both simulation studies (LeBreton et al. 2004b) and primary studies (LeBreton et al. 2004a; LeBreton et al. 2007). Recently, new developments in relative weight analysis have expanded its applications from traditional multiple regression to more complicated regression models such as multivariate multiple regression (LeBreton and Tonidandel 2008), regression models containing higher order terms such as cross-product terms, quadratic terms, or other polynomial terms (LeBreton et al. 2013), logistic regression (Tonidandel and LeBreton 2010), and multivariate analysis of variance (MANOVA; Tonidandel and LeBreton 2013). In addition to obtaining the relative weights themselves, more useful information can be gleaned by supplementing the weights with confidence intervals, testing the statistical significance of the weights, and evaluating the statistical significance of the difference between weights either within or between samples (Johnson 2004; Tonidandel et al. 2009).

Applications of Relative Weight Analysis

Relative weight analysis has been used across a variety of domains in the organizational sciences literature. For example, Lopina et al. (2012) used relative weight analysis to examine the relative importance of individual difference variables for predicting turnover. In a related vein, Major et al. (2013) used relative weight analysis to understand how men and women differ in terms of the most important predictors of organizational commitment. Turning to leadership, Braddy et al. (2013) examined the relative importance of different leader behaviors for predicting derailment, whereas Snell et al. (2013) used a similar approach to understand how political skill predicts managerial effectiveness. Concerning job performance, Dalal et al. (2012) examined the relative importance of employee engagement, personality, and job attitudes as predictors of performance, and Gentry et al. (2013) used relative weight analysis to understand how different character virtues might be more or less important for managers at different levels of an organization. Scherer et al. (2013) used relative weight analysis to examine the contributions of sub-clinical psychopathy and the global traits of the Five Factor Model in the prediction of counter-productive workplace behaviors. As a final example, Lievens and Peeters (2008) used relative weight analysis to understand how different impression management tactics are related to evaluations of interviewees. As is evident from these examples, relative weight analysis is applicable to many different areas of research.

RWA-Web

To assist those wishing to perform relative weight analysis in their own work, we endeavored to create a more general solution that would integrate the disparate types of relative importance calculations within a single easy-to-use framework that would be available to all researchers regardless of their preferred platform or fluency with a particular platform. To that end, we created an interactive website that performs all of these calculations using the free, open-source R statistical package as its backbone. The website is designed such that users who are familiar and comfortable with R can use the website to generate the R code necessary to conduct the various RWAs. These users can simply download the code and execute it locally on their workstation.

However, many individuals may not be familiar with R (and may not want to learn R). Consequently, we have also designed the website so that users may simply point-and-click through the website to generate the R syntax and then submit the syntax and a data file to our R web server. This server will then run the desired analyses and send the results to the user via e-mail. Thus, users can successfully conduct these relative importance analyses with no prior knowledge of R or experience using R.

Visiting the Website and Selecting a RWA Model

RWA-Web is available at: http://relativeimportance.davidson.edu/. This website will continue to be updated as new procedures relevant to RWA are developed. In its current form, RWA-Web permits researchers to examine the relative importance of multiple (correlated or uncorrelated) predictors using (1) a standard multiple regression model, (2) a multivariate multiple regression model containing multiple (correlated or uncorrelated) dependent variables, and (3) a standard logistic regression model. A screen shot from the landing page is presented in Fig. 1. From this landing page, users simply select which type of RWA analysis they would like to perform. Each procedure has a separate page that contains instructions and an easy-to-use interface for conducting the relative importance analyses (Fig. 2). In addition, there is a list of references for the user seeking additional information about relative weights (in general) or the specific options available through RWA-Web.

Fig. 1
figure 1

RWA-Web landing page (http://relativeimportance.davidson.edu)

Full size image
Fig. 2
figure 2

RWA-Web Form for traditional relative weight analysis (http://relativeimportance.davidson.edu/multipleregression.html)

Full size image

This website also contains a link to the data file used for the illustrative examples in the current paper. Readers wanting to confirm their understanding of relative weight analysis and the RWA-Web tool are encouraged to replicate our findings using these data. RWA-Web was designed with three key features in mind: (1) ease of use, (2) inclusiveness, and (3) flexibility. Below, we briefly comment on these features before presenting a tutorial on using RWA-Web including the interpretation of results obtained using several illustrative datasets.

Ease of Use

The interface of each page is designed as an editable web form where users can select certain options by choosing various check boxes and provide additional detailed information in text boxes. For each type of analysis (multiple regression, multivariate regression, and logistic regression), the user only needs to provide three basic pieces of information in order to obtain results: the name and location of the data file, the name of the criterion variable(s), and the name of the predictor variables (note that R is case sensitive). The default file format for the data file is a comma-separated values (.csv) file, which is a standard exporting option in Excel and most other statistical software packages. With these minimal pieces of information, the website will generate raw and rescaled relative weights for each predictor (scaled as the percentage of explainable variance), the bootstrapped confidence intervals around these weights, and the corresponding tests for significance of these weights.

While disparate resources exist for calculating relative weights, this is the first platform to integrate the relative weight results with confidence intervals and tests of statistical significance. Moreover, the only other program currently available for calculating confidence intervals does not rely on the bias corrected accelerated method of bootstrapping, which was recommended by Tonidandel et al. (2009) after extensive simulation work.

The ability to easily obtain confidence intervals and tests of significance will hopefully improve the accessibility of relative importance statistics within the organizational sciences. Without having this information readily available, researchers have tended to rely on the flawed practice of merely looking at the rank ordering of predictor importance without taking into account the sampling distributions of these statistics (Tonidandel and LeBreton 2011). By focusing solely on rank order, one might incorrectly conclude a predictor to be more important than another predictor despite a considerable degree of overlap in their respective sampling distributions. Thus, by having the additional information of confidence intervals and tests of significance readily available, one can make more informed judgments regarding the true contribution of predictors to explainable variance.

Inclusiveness: Built-in Comprehensive Functionality

In addition to integrating the relative weights, confidence intervals, and tests of statistical significance, within a single platform, the website also possesses crucial additional features to make it a truly comprehensive resource for these analyses. Frequently, in addition to wanting to determine the relative importance of various predictors, researchers often have hypotheses that might predict that two predictors differ significantly from one another in terms of their relative importance or that a particular predictor’s relative importance differs in two different samples. To address either of these questions, one needs to be able to evaluate the statistical significance of the difference between relative weights either within or between samples. This new resource can perform both of these types of tests using bias corrected accelerated methods. To test for the statistical significance of the difference between relative weights within a sample, a user only needs to supply the name of a predictor variable, and the relative weight for that predictor will be tested against all of the other predictors in the model. To evaluate the statistical significance between samples, one simply provides the name of the grouping variable that identifies the different samples within the dataset and the two levels of that grouping variable one wishes to compare. The program will then automatically generate tests for significance for all of the predictors in the model between the two samples.

Flexibility

We have designed the website in an effort to provide a maximum amount of flexibility to accommodate the needs of various users. To that end, the point-and-click interface allows users to select from a variety of options including: conducting relative importance analyses on both raw data or on a correlation matrix, choosing the alpha value for the confidence intervals and tests of statistical significance as well as the number of bootstrapped replications they wish to use for those analyses, and electing different techniques for handling missing data should they have any. Finally, individuals comfortable with the R statistical program can elect to download the R code and can then modify it to meet their own personal needs.

Illustrative Example—Standard Multiple Linear Regression

To illustrate the features of RWA-Web, we will use a simulated dataset containing a sample of 604 employees. The employees completed a climate survey measuring: Customer Orientation (custom: higher scores = more customer orientation), Team Orientation (team: higher scores = greater emphasis on teamwork), Diversity Orientation (diverse: higher scores = greater emphasis on promoting ethnic and gender diversity in the workplace), Safety Orientation (safety: higher scores = greater emphasis on maintaining a safe work environment), and Communication (comm: higher scores = more effective communication from the organization’s leaders to the employees). Three months later, the same employees completed a job attitude survey measuring: Organizational Commitment (commit: higher scores = stronger commitment to the organization), Job Satisfaction (jobsat: higher scores = higher overall levels of job satisfaction), and Leader Effectiveness (ldr_eff: higher scores = higher overall perceptions of effectiveness). Finally, one year after the initial survey, data were collected concerning voluntary turnover from the organization (turnover: 0 = employee remained, 1 = employee attrited).

Using RWA-Web to Estimate Relative Weights

Using these data, we sought to examine the relative importance of the climate variables in predicting Job Satisfaction, Organizational Commitment, and Leader Effectiveness. Specifically, we wanted to estimate relative weights (Johnson 2000) along with bootstrapped standard errors for use with significance testing (Tonidandel et al. 2009).

To perform these analyses we selected the “Multiple Regression” link from the home page (see Fig. 1). This took us to the RWA-Web page for traditional multiple regression. From here, we completed the web form as follows:

  1. (1)

    Rather than downloading the R code and running it locally, we opted to “Send to R Server.”

  2. (2)

    Consequently, we needed to specify a dataset that would be sent to the R Server along with the R Code that we would generate in subsequent steps. Our data were originally saved as an SPSS “.sav” data file. However, we converted this file into a “.csv” file by opening the data file in SPSS, selecting “File” and “Save as” from the drop down menus. In the new dialog box, we selected “Comma delimited (*.csv)” under the “Save as type” options.

  3. (3)

    Because we were sending the R code and data to the R server, we also entered an e-mail address to which the results could be sent. We noted that the data were “Raw data” and that we wanted to delete data using “Pairwise deletion,” even though the dataset had complete data.

  4. (4)

    Next, we specified the first relative weight analysis. We entered “jobsat” as our criterion variable and entered the names of our climate variables (i.e., custom, team, diverse, safety, comm) as our predictor variables (hitting “enter” or “return” after each variable was entered in the list).

  5. (5)

    We retained the default bootstrapping options (k = 10,000 replications; alpha = 0.05). We also requested a comparison of whether the relative importance of the “safety” variable differed significantly from the remaining predictor variables. That is, we sought to test a hypothesis that the “safety” variable was statistically larger than all of the remaining predictor variables.

  6. (6)

    We also requested an analysis comparing whether the relative weights differed across the “gender” variable.

  7. (7)

    After making these selections, we clicked “submit” and waited for the e-mail from the R server. We received the e-mail about 10 min later.

  8. (8)

    Finally, we repeated steps 1–6 for both the “commit” and “ldr_eff” outcome variables.

Interpreting the Output from RWA-Web

Table 1 contains the abbreviated output for the job satisfaction criterion. Specifically, we copied and pasted the results portion of the e-mail into Table 1. We did not bother copying and pasting the earlier portion of the e-mail that includes the R code for these procedures. We now provide a brief interpretation of the results from each section of the abbreviated output. It is also important to acknowledge that researchers will not be able to perfectly reproduce the results discussed in our paper because those results are based on a bootstrapping procedure. However, substantive conclusions should be generally consistent, and the numeric values in our tables should be fairly similar to those that would be obtained as others replicate our analyses.

Table 1 RWA-Web: results for a traditional relative weight analysis (criterion = job satisfaction)
Full size table

#The Raw and Rescaled Weights

This section contains a summary of the relative importance weights for each of the climate variables. The column labeled “Raw.RelWeight” provides estimates of variable importance using the metric of relative effect sizes (LeBreton et al. 2007). Specifically, these weights represent an additive decomposition of the total model R 2 and can be interpreted as the proportion of variance in job satisfaction that is appropriately attributed to each climate variable. For example, communication explained ~18 % of the variance in job satisfaction, whereas diversity orientation and safety orientation each explained ~4 % of the variance in job satisfaction. By summing the raw weights we obtain the total model R 2 = 0.49 (i.e., 0.1234 + 0.1022 + 0.0467 + 0.0410 + 0.1789 = 0.4922), which matches the R 2 provided in the first line of the output.

The values listed under the “Rescaled.RelWeight” column were obtained by dividing each raw relative weight by the model R 2. These rescaled weights provide estimates of relative importance using the metric of percentage of predicted variance attributed to each variable. For example, communication explains ~36 % of the predicted variance in job satisfaction. That is, communication explains ~36 % of the total R 2 (i.e., 0.1789/0.4922 = 0.3635 × 100 = 36.35 %). The difference between the raw weights and the rescaled weights is simply in the interpretation one draws. Thus, it is important for researchers to be clear as to whether they are reporting raw weights based on the metric of relative effect sizes or rescaled weights based on the metric of percentage (or proportion) of predicted or explained variance in the criterion that should be attributed to each predictor.

#BCa Confidence Intervals around the Raw Weights

The next section of output provides confidence intervals (CIs) around the raw weights. These CIs are useful for explaining the precision of the raw relative weights. Larger CIs indicate less precision; smaller CIs indicate greater precision. Because relative weights can never be zero,Footnote 1 the CIs will tend to be positively skewed, especially for relative weights that are close to zero. As noted above, we retained the alpha value of 0.05, thus these represent 95 % CIs around the point estimates for the raw relative weights.

#BCa Confidence Interval Tests of Significance

Whereas the prior section of output provides confidence intervals that can be used to evaluate the precision of the sample raw relative weight, the next two sections provide information about the statistical significance of the raw relative weights obtained by calculating bias corrected and accelerated CIs as described by Tonidandel et al. (2009). Briefly, because raw weights are scaled in terms of relative effect sizes (i.e., variance explained) they cannot be zero (or negative; Johnson 2004; Tonidandel et al. 2009). Thus, Tonidandel et al. describe a procedure that compares the observed relative weights to the relative weight obtained by including a randomly generated variable in the regression analysis. Because the variable is completely random, it has a population relative weight of zero. Consequently, it is possible to test whether the observed relative weights significantly differ from the relative weight for the random variable. The logic of their procedure is roughly analogous with the logic of using random data to determine the number of factors in a dataset via parallel analysis. An examination of the CIs immediately following the section labeled “#If Zero is not included, Weight is Significant” reveals that all of the raw weights presented in the first part of the output are statistically significant.

The next section of output also involves significance testing, but this section is focused on “#Comparing one predictor with all others.” Specifically, we requested a comparison of whether the raw relative weight for safety orientation was significantly different from the relative weights for the remaining climate variables. As noted in the output, if zero is included in the BCa CI, then the two weights are not significantly different from one another. In our example, we see that safety orientation is significantly different from customer orientation, team orientation, and communication. However, it is not different from diversity orientation. An intuitive examination of the raw weights confirms these results. We see that both diversity orientation and safety orientation have raw weights of ~0.04. And, we see that the weights for the other climate variables are much larger than the weight for safety.

#The Raw and Rescaled Weights for Gender

The next section of output presents the raw and rescaled relative weights separately for each level of our gender variable. Recall, we requested a comparison of whether the relative weights for our climate variables differed across levels of employee gender. The final section of output, “#Comparing predictors across 2 groups” presents the BCa CIs comparing the differences in raw weights as a function of employee gender. As noted on the output, “#If Zero is not included, weights are significantly different between the groups.” An examination of this final section of output indicates that there were no statistically significant differences in the raw relative weights when comparing men to women. This analysis allows one to examine whether the relative importance of particular predictors differs as a function of gender. This approach is akin to testing the difference in regression weights using moderated multiple regression, but the focus here is on relative importance instead of prediction.

Summary

Table 1 provides substantial information, but how should a researcher go about integrating that information and presenting it in his or her article or technical report? Although there is not a single best way to present results, we have extracted the key elements from the output and assembled them in Table 2. And, based on these results, a researcher might say something along the lines of:

Table 2 Summary of a traditional relative weight analysis
Full size table

“A relative weight analysis (Johnson 2000) was conducted using RWA-Web (Tonidandel and LeBreton 2014); results from this analysis are summarized in Table 2. Confidence intervals for the individual relative weights (Johnson 2004) and all corresponding significance tests were based on bootstrapping with 10,000 replications, an approach recommended by Tonidandel et al. (2009). Bias corrected and accelerated confidence intervals were used because of their superior coverage accuracy as recommended by Tonidandel et al. (2009). In all cases, 95 % CIs were used (corresponding to a significance testing alpha level of 0.05). Briefly, these results indicate that a weighted linear combination of our five climate variables explained roughly half of the variance in the job satisfaction criterion (R 2 = 0.49). An examination of the relative weights revealed that all five variables explained a statistically significant amount of variance in job satisfaction as none of the 95 % CIs for the tests of significance contained zero, with the most important variables being Customer Orientation (RW = 0.12), Team Orientation (RW = 0.10), and Communication (RW = 0.18).

The relative weight results differ slightly from what was obtained from the traditional multiple regression analysis. Specifically, in the traditional analysis neither Diversity Orientation nor Safety Orientation provided a statistically significant incremental effect in the prediction of Job Satisfaction, holding constant all of the remaining climate variables. The lack of concordance in the significance of the regression coefficients and the relative weights is not uncommon (Tonidandel et al. 2009) and simply reflects that these two statistics are addressing different research questions. Regression weights are focused on incremental prediction and when predictors are correlated, variables that yield a significant bivariate relationship may not yield a significant incremental relationship; relative weights are focused on explaining which predictors are explaining non-trivial variance in our outcomes (even in the presence of additional, correlated predictors). Thus, these results work in a supplementary fashion (Tonidandel and LeBreton 2011) and suggest that both of these variables are explaining non-trivial variance in Job Satisfaction, but the correlations they share with one another (and the other climate variables) results in them explaining little unique, incremental variance.

We were also interested in testing whether the relative contribution of safety orientation to the overall R 2 was significantly different from the remaining climate variables. Results indicated that relative weight of safety orientation (RW = 0.04) was significantly lower than all of the remaining climate variables (i.e., CIs for the comparisons did not include zero), except diversity orientation (RW = 0.05). Finally, we also tested whether there were differences in the magnitude of the relative weights as a function of employee gender. Results indicated there were no statistically significant differences as a function of gender; the CI for the male–female comparison did include zero. In sum, it appears that the majority of the explained/predicted variance in job satisfaction is attributed to customer orientation (25 % of model R 2), team orientation (21 % of model R 2), and communication (36 % of model R 2).”

Table 2 also includes the results for the other two outcome variables (i.e., organizational commitment and leader effectiveness). We leave obtaining these results via RWA-Web as an exercise for the interested reader.

Using RWA-Web for Other Procedures

Using the dataset described under Example #1, one could very easily execute relative weight analyses using both multivariate multiple regression and logistic regression. For multivariate relative weight analysis, one would simply select “Multivariate Regression” from the main landing, and the same general options could be selected from the web form with the exception of including multiple criterion variables (i.e., jobsat, commit, ldr_eff) in the analysis.

Whereas the traditional relative weight analysis yields raw weights that sum to R 2, the multivariate analysis yields weights that sum to a multivariate analog of R 2. LeBreton and Tonidandel (2008) reviewed several potential multivariate analogs of R 2 and RWA-Web estimates multivariate weights using one of the recommended analogs: \( P^{2}_{\text{yx}} \). The content and structure of output mirrors that of Table 1.

For logistic relative weight analysis (Tonidandel and LeBreton 2010), one could examine the relative importance of our five climate variables in predicting the “turnover” variable in our example dataset by simply selecting “Logistic Regression” from the main landing page.

Similar to the multivariate analyses, the estimation of logistic relative weights necessitates the identification of a logistic analog to R 2. RWA-Web estimates logistic relative weights using one of the recommended analogs: \( R^{2}_{\text{O}} \). The content and structure of the results again mirrors that of Tables 1.

Conclusion

In conclusion, we hope that this article has documented how easy it is for scholars to integrate relative weight analysis into their research activities. We believe that relative importance analyses provide important supplemental information that should be regularly included along with the results typically provided when using regression analyses.

The typical indices produced by regression are useful but do not accurately partition variance among correlated predictors, whereas relative weight analysis is properly suited for this function. Moreover, relative importance analyses provide valuable information to researchers who can address substantive hypotheses that are not easily addressed by traditional regression results, and because they are scaled in the metric of variance explained, importance weights may be interpreted as estimates of relative effect size. RWA-Web fills an important void by providing scholars with the tools to conduct relative importance analyses in a user-friendly format irrespective of the statistical platform to which one is most accustomed. Thus, with minimal effort, a researcher can conduct relative importance analyses as a supplement to their planned regression analyses.