Moderators in meta-analysis: subgroup analysis vs meta-regression

When the effect sizes of a meta-analysis differ more than chance would explain, the natural next question is why. A moderator is a study-level characteristic that may explain part of that variation, such as the population studied, the type of intervention, the measure used, or the year of publication. Two techniques test moderators: subgroup analysis and meta-regression. They answer the same question with different tools (Borenstein et al., 2021).

Subgroup analysis: comparing groups of studies

In a subgroup analysis the studies are split by a categorical moderator (for example, one version of an intervention against another), a pooled effect is computed in each subgroup, and the subgroup means are compared. It is the meta-analytic counterpart of an analysis of variance (Borenstein et al., 2021).

• With two subgroups the comparison can be a Z-test; with two or more, a Q-test that splits the total heterogeneity into a between-subgroups component and a within-subgroups component (see the formula in Methods and formulas). The procedures are mathematically equivalent (Borenstein et al., 2021).
• The question being tested is whether the subgroup means differ, not whether each subgroup shows an effect.
• These tests require independent subgroups: the same participants must not contribute to more than one subgroup (Higgins et al., 2019).

Meta-regression: a regression across studies

Meta-regression extends the idea to continuous moderators (mean age, dose, year) and, in principle, to several moderators at once. It is the meta-analytic counterpart of regression, with two differences: the outcome is the study's effect size, and the studies are weighted by the precision of their estimates, with allowance for residual heterogeneity in a random-effects meta-regression (Higgins et al., 2019).

• The regression coefficient shows how much the effect size changes for a one-unit increase in the moderator. For ratio measures such as odds ratios the analysis is run on the log scale (Higgins et al., 2019).
• A categorical moderator can also be entered in a meta-regression, using dummy variables; with J subgroups, J − 1 dummies, each estimating the difference from a reference subgroup (Higgins et al., 2019).

How to read the results

• Is there a difference? Look at the test of the difference between subgroups (or at the coefficient's test in meta-regression).
• How big is it? Significance is not enough. The proportion of the true variance explained by the moderator is the analogue of R² in primary studies, and it should be reported together with the test (Borenstein et al., 2021). For subgroup differences the Cochrane Handbook also reports an I² statistic (Higgins et al., 2019).
• Is it relevant? If a difference would not change any conclusion or recommendation, the overall result may be the better thing to present (Higgins et al., 2019).

What to report

• Which moderators were tested, why, and whether they were pre-specified.
• The number of studies in each subgroup, or the number analysed in the meta-regression.
• The test of the difference, the effect in each subgroup or the regression coefficient with its confidence interval, and the proportion of variance explained.
• A cautious interpretation, as a hypothesis for future research rather than a causal finding.

Learn this in depth

The pitfalls of moderator analyses (low power, confounding, too few studies, post hoc searching), choosing between subgroup analysis and meta-regression, estimating τ² in each subgroup, recoding sparse categories, building multiple-moderator models and reading their output in practice are covered step by step in the ActiLab courses and manual, currently in preparation. Join the newsletter to be notified.

For the place of this analysis in the workflow, see Testing moderators; for the heterogeneity it tries to explain, see Heterogeneity in meta-analysis; for the number of studies needed, see How many studies do you need?.

References

Book Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to meta-analysis (2nd ed.). Wiley.

Book Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2019). Cochrane handbook for systematic reviews of interventions (2nd ed.). John Wiley & Sons. Handbook updated online available at https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current