Meta-analyses: standardised mean differences

Answers

Statement c is true, whereas a, b, and d are false.

The meta-analysis investigated the effects of local anaesthesia for pain control during outpatient hysteroscopy. Randomised controlled trials were included if they compared local anaesthesia with control treatment—no intervention, placebo, oral analgesics, or conscious sedation. In total, 15 trials were identified. The purpose of the meta-analysis was to combine the sample estimate of the treatment effect on pain control from each trial to give a subtotal estimate for each method of administration plus a total for all methods combined, thereby reducing a large amount of information to a manageable quantity. For each trial, the treatment effect was the difference between the local anaesthesia and control groups in the assessment of pain. The results of the meta-analysis for all four subgroups of administration of local anaesthesia—intracervical, paracervical, and transcervical injections plus topical application—were presented (fig 2⇓). The forest plot shows the subtotal estimates for each method of administration, plus the total estimate across all groups. A previous question described how to read a forest plot.2

Fig 2 Effect of local anaesthesia compared with control treatment on pain during outpatient hysteroscopy, according to method of administration

• Download figure
• Open in new tab
• Download powerpoint

The primary outcome was pain associated with hysteroscopy. The trials used different continuous scales to assess pain. A meta-analysis of an outcome measured on a continuous scale relies on the outcome being assessed using the same scale in all of the trials. Therefore, in the example above, it was not possible to combine the sample estimates directly to derive a subtotal estimate for each method of administration plus a total estimate for all methods combined. To account for the differences between trials in the scales used to assess pain, the standardised mean difference between treatment groups was derived for each trial and used to derive the subtotal and total estimates of the treatment effect.

For each trial, the standardised mean difference was calculated as the mean difference between treatment groups divided by the standard deviation of the assessment of pain for all participants pooled across both treatment groups. It is commonly incorrectly stated that the standard error, rather the standard deviation, is used to calculate the standardised mean difference (a is false). The standardised mean difference expresses the difference between treatment groups in the assessment of pain as multiples of the observed standard deviation. The standardised mean difference is a ratio, with numerator and denominator in the same units as the original measurement. It therefore has no units, does not depend on the original measurement scale (b is false), and allows a direct comparison of the treatment effect across trials that used different scales to assess pain (c is true). Standardised mean differences have similar properties to those of effect sizes, described in a previous question.3

The subtotal and total estimates in the above meta-analysis were based on standardised mean differences; they are not easy to interpret and may have limited value as a measure of treatment effect. These estimates may serve only as quantitative measures of the strength of evidence against the null hypothesis of no difference in the population between local anaesthesia and control treatments in pain control during outpatient hysteroscopy. The larger the absolute value of the subtotal and total estimates, the greater the treatment effect between local anaesthesia and control treatments.

The forest plot was plotted on a linear scale. Therefore, for the sample estimates in each trial plus the subtotal and total estimates the 95% confidence intervals are displayed as symmetrical around the standardised mean difference. For a meta-analysis based on mean differences, actual or standardised, the vertical line through zero represents the line of no treatment effect.

For the trials included in the meta-analysis, higher scores on the pain assessment scales used indicated greater pain associated with hysteroscopy. For each trial, the standardised mean difference was calculated by subtracting the mean score for the control group from the mean pain score for the local anaesthesia treatment group. Therefore, as indicated on the forest plot, a standardised mean difference less than zero favoured local anaesthesia, whereas one above zero favoured control (fig 2). The researchers reported that pain in women undergoing hysteroscopy as outpatients was significantly reduced by intracervical (standardised mean difference −0.36, 95% confidence interval −0.61 to −0.10) and paracervical (−1.28, −2.22 to −0.35) injections of local anaesthesia (d is false). However, no significant difference was found between local anaesthesia and control for transcervical injections (−0.11, −0.31 to 0.10) and topical application (−0.32, −0.97 to 0.33).

The standardised mean differences between the randomised groups were weighted before deriving the subtotal and total estimates. Standardised mean differences are sometimes referred to as weighted standardised mean differences. The weight for a trial indicates how much influence that trial had on the subtotal and total estimates for the meta-analysis. The weights given to the trials in deriving the total estimate, regardless of method of administration, are shown in the column headed “Weight (%)” (fig 2). The weight given to a standardised mean difference for a trial was determined by the precision of the estimate of the treatment effect—trials with more precise estimates had greater weight. In a meta-analysis, the precision of an estimate for a trial is usually represented by the inverse of the variance of the outcome pooled across all participants in the trial. Less precise estimates have larger variances, so the inverse of variance is smaller for trials with less precise estimates.