ActiLab.eu

How many studies do you need for a meta-analysis?

Short answer: there is no universally accepted minimum. Two studies can be combined, but what you can conclude depends on what you want to estimate. The number that is "enough" is different for a pooled effect, for heterogeneity, and for moderators.

Two studies: possible, but not the whole story

Technically, a meta-analysis needs only two studies. Valentine, Pigott, and Rothstein (2010) argue that, given the need to reach a conclusion, two studies are enough, because the alternatives (vote counting, comparing effect sizes by eye) are less transparent and less likely to be valid. They add important caveats: with very few studies, estimates of the average effect and of the between-study variance will be poor and conclusions highly uncertain, and a meta-analysis of two studies is likely to be informative only if they are close replications of one another. Combining very few, very different studies is rarely tenable.

The pooled effect: power depends on more than k

The statistical power of a meta-analysis is not set by the number of studies alone. It depends on:

Heterogeneity costs power under random effects. In the worked example of Valentine et al. (2010), a review expecting a small effect (δ = 0.15) from studies with 20 participants per group would need about 27 studies for 80% power under a fixed-effect model, but about 55 under random effects with moderate heterogeneity. These numbers depend entirely on the example's assumptions and are not general benchmarks.

Jackson and Turner (2017) showed that, under random effects, a meta-analysis can even be less powerful than the individual studies it combines, because the between-study variance must be estimated. In an analysis of 1991 Cochrane meta-analyses, this was still common with few studies, and they concluded that roughly five or more studies are needed for the meta-analysis to reasonably consistently outperform its component studies; with fewer, random-effects inference is difficult and of limited added value. The practical consequence is that a power analysis, following Hedges and Pigott (2001) and Valentine et al. (2010), is more informative than any fixed threshold.

Heterogeneity: be cautious with small k

Estimates of between-study variance (τ²), and of statistics built on it such as I² and the prediction interval, are unstable when there are few studies. There is no agreed cut-off. As a rough indication, Borenstein, Hedges, Higgins, and Rothstein (2021) suggest that ten studies may serve as a useful minimum in many cases, and note that as many as 20 may be needed to estimate τ² reliably. With few studies, report heterogeneity statistics with their uncertainty, and avoid reading too much into a low I².

Moderators: the "ten studies" rule of thumb

For moderator analyses (subgroup analysis and meta-regression), the Cochrane Handbook (Higgins et al., 2019) advises that at least ten studies should be available for each characteristic modelled, and warns that even this is too few when the characteristic is unevenly distributed across studies. It is borrowed from advice for simple regression analyses, so it is a rule of thumb, not a derived threshold; Borenstein et al. (2021) likewise stress that there are no hard and fast rules. Moderator tests have lower power than the main effect (Hedges & Pigott, 2004; Valentine et al., 2010), so a null result with few studies is not evidence that the moderator does not matter. Every additional moderator also consumes information, which increases the risk of overfitting.

Publication bias: checks need enough studies

Tests for small-study effects and publication bias are also unreliable with few studies. As a rule of thumb, the Cochrane Handbook recommends using tests for funnel plot asymmetry only when at least 10 studies are included, because their power is low with fewer; see Evaluating publication bias.

What to do with a small number of studies

Learn this in depth

Choosing between model assumptions, estimating τ² reliably, planning power, and deciding how many moderators a dataset can support involve trade-offs that go beyond rules of thumb. These topics are covered step by step in the ActiLab courses and manual, currently in preparation. Join the newsletter to be notified.

References

Book Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to meta-analysis (2nd ed.). Wiley.

Article Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6(3), 203–217. https://doi.org/10.1037/1082-989X.6.3.203

Article Hedges, L. V., & Pigott, T. D. (2004). The power of statistical tests for moderators in meta-analysis. Psychological Methods, 9(4), 426–445. https://doi.org/10.1037/1082-989X.9.4.426

Book Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Welch, V. A. (Eds.). (2019). Cochrane handbook for systematic reviews of interventions (2nd ed.). John Wiley & Sons. Handbook updated online available at https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current

Article Jackson, D., & Turner, R. (2017). Power analysis for random-effects meta-analysis. Research Synthesis Methods, 8(3), 290–302. https://doi.org/10.1002/jrsm.1240

Article Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010). How many studies do you need? A primer on statistical power for meta-analysis. Journal of Educational and Behavioral Statistics, 35(2), 215–247. https://doi.org/10.3102/1076998609346961