Evaluating publication bias

Alongside moderator testing, the full set of study effect sizes is also used to check for publication bias, which exists when published studies (those that can be easily retrieved) differ systematically from unpublished studies (gray literature). In meta-analysis, the potential impact of publication bias can be assessed through different methods:

• Funnel plot
• Egger's linear regression method
• Begg and Mazumdar's rank correlation method
• Duval and Tweedie's Trim and Fill method
• Rosenthal's Fail-safe N

The fail-safe N estimates how many unpublished null-result studies would be needed to overturn a significant overall effect; a conventional rule of thumb treats it as reassuring once it exceeds 5k + 10 (where k is the number of studies included). It's widely reported but also widely criticized — it assumes missing studies would average out to exactly zero effect, and it ignores heterogeneity and sample size — so it's best used alongside the funnel-plot-based methods above rather than on its own. No single method is conclusive: converging results across several of them is what actually supports a judgment that publication bias is minimal, moderate, or substantial.

The result of this step is that judgment — an assessment of how much confidence the overall effect size deserves once missing studies are accounted for, to be reported alongside the results in Publishing a meta-analysis.