Meta-analysis methods and formulas

This page collects the most common formulas behind each step of a meta-analysis described in Meta-analysis steps — enough to understand how the numbers are actually computed, not an exhaustive reference. ProMeta implements a much wider range of formulas, covering many more effect-size types, models, and edge cases than shown here. In all formulas below, k is the number of studies, and each study i contributes an effect size ( $Y_{i}$ ) and its variance ( $v_{i}$ ).

Each study's effect size and weight feed a pooled estimate; heterogeneity, moderators, and publication bias are then checked around that pooled estimate

Effect sizes based on means

Cohen's d standardizes the raw difference between two group means using the pooled standard deviation:

\begin{matrix} d = \frac{M_{1} - M_{2}}{{SD}_{pooled}} \\ \begin{array}{l} d = Cohen's d \\ M_{1} = mean of group 1 \\ M_{2} = mean of group 2 \\ {SD}_{pooled} = pooled standard deviation \end{array} \end{matrix}

\begin{matrix} {SD}_{pooled} = \sqrt{\frac{(n_{1} - 1) {SD}_{1}^{2} + (n_{2} - 1) {SD}_{2}^{2}}{n_{1} + n_{2} - 2}} \\ \begin{array}{l} {SD}_{pooled} = pooled standard deviation \\ n_{1}, n_{2} = sample sizes of group 1 and group 2 \\ {SD}_{1}, {SD}_{2} = standard deviations of group 1 and group 2 \end{array} \end{matrix}

Cohen's d is slightly biased in small samples. Hedges' g corrects for this with a small-sample correction factor J:

\begin{matrix} g = d \times J, J = 1 - \frac{3}{4 df - 1} \\ \begin{array}{l} g = Hedges' g (bias-corrected d) \\ d = Cohen's d \\ J = small-sample bias-correction factor \\ df = degrees of freedom \end{array} \end{matrix}

The variance of d (needed to weight the study in the pooled analysis) is:

\begin{matrix} v_{d} = \frac{n_{1} + n_{2}}{n_{1} n_{2}} + \frac{d^{2}}{2 (n_{1} + n_{2})} \\ \begin{array}{l} v_{d} = variance of Cohen's d \\ n_{1}, n_{2} = sample sizes of group 1 and group 2 \\ d = Cohen's d \end{array} \end{matrix}

Effect sizes based on binary data

For a 2×2 table of a study (a, b, c, d = cell counts), the log odds ratio and its variance are:

\begin{matrix} \ln (OR) = \ln \frac{a d}{b c} \\ \begin{array}{l} OR = odds ratio \\ a, b, c, d = 2×2 table cell counts: a = exposed with event, b = exposed without event, c = unexposed with event, d = unexposed without event \end{array} \end{matrix}

\begin{matrix} v_{\ln (OR)} = \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} \\ \begin{array}{l} v_{\ln (OR)} = variance of the log odds ratio \\ a, b, c, d = 2×2 table cell counts, same as in the odds ratio above \end{array} \end{matrix}

The log risk ratio, computed on the same table, is:

\begin{matrix} \ln (RR) = \ln \frac{\frac{a}{a + b}}{\frac{c}{c + d}} \\ \begin{array}{l} RR = risk ratio \\ a, b, c, d = 2×2 table cell counts: a = events in the exposed group, b = non-events in the exposed group, c = events in the unexposed group, d = non-events in the unexposed group \end{array} \end{matrix}

Both are analyzed on the log scale (where sampling distributions are closer to normal) and only exponentiated back to OR/RR for reporting.

Effect sizes based on correlations

Pearson's r is not analyzed directly, since its sampling distribution is skewed for values far from zero. It is first converted to Fisher's Z:

\begin{matrix} Z_{r} = 0.5 \times \ln \frac{1 + r}{1 - r} \\ \begin{array}{l} Z_{r} = Fisher's z-transformed correlation \\ r = Pearson correlation coefficient \end{array} \end{matrix}

\begin{matrix} v_{Z_{r}} = \frac{1}{n - 3} \\ \begin{array}{l} v_{Z_{r}} = variance of the Fisher-transformed correlation \\ n = sample size \end{array} \end{matrix}

Pooled results are converted back to the r scale only at the end, for reporting.

Once an effect size is computed for every study, they still need to be combined into a pooled estimate — see Aggregating effect sizes and Multilevel models for the models that do this, and the weights ( $w_{i}$ ) and pooled estimate (M) referenced below.

Heterogeneity

Q tests whether the observed variation across study effect sizes is larger than expected from sampling error alone (its null distribution is chi-squared with k − 1 degrees of freedom):

\begin{matrix} Q = \sum_{i = 1}^{k} w_{i} {(Y_{i} - M)}^{2} \\ \begin{array}{l} Q = Cochran's Q, the heterogeneity statistic \\ w_{i} = weight given to study i \\ Y_{i} = effect size of study i \\ M = pooled (overall) effect size \\ k = number of studies \end{array} \end{matrix}

Q is sensitive to the number of studies, so I² is used to quantify heterogeneity as a percentage of total variation, independent of k:

\begin{matrix} I^{2} = \max (0, \frac{Q - df}{Q}) \times 100 % \\ \begin{array}{l} I^{2} = proportion of total variance due to real heterogeneity \\ Q = Cochran's Q, the heterogeneity statistic \\ df = degrees of freedom (k − 1) \end{array} \end{matrix}

As a rule of thumb (Higgins & Thompson, 2002), I² around 25%, 50%, and 75% are considered low, moderate, and high heterogeneity — though this should always be read alongside the confidence interval around I², not as a hard cutoff.

Moderators

When a categorical moderator splits studies into p subgroups, Q decomposes into a between-subgroups component, comparing each subgroup's pooled mean to the overall mean M, and a within-subgroups component (how much heterogeneity remains inside each subgroup):

\begin{matrix} Q_{B} = \sum_{g = 1}^{p} w_{g} {(M_{g} - M)}^{2} \\ \begin{array}{l} Q_{B} = between-subgroup heterogeneity statistic \\ w_{g} = weight given to subgroup g \\ M_{g} = pooled effect size of subgroup g \\ M = overall pooled effect size \\ p = number of subgroups \end{array} \end{matrix}

A continuous moderator is tested instead with a weighted meta-regression, where the effect size is regressed on the moderator using the same inverse-variance weights as the pooled model above.

Publication bias

Rosenthal's Fail-safe N estimates how many additional, unpublished null-result studies — each contributing a $Z_{i}$ of 0 — would need to exist to bring a significant pooled result down to non-significance:

\begin{matrix} N_{fs} = \frac{{(\sum_{i = 1}^{k} Z_{i})}^{2}}{2.706} - k \\ \begin{array}{l} N_{fs} = Rosenthal's fail-safe N \\ Z_{i} = standard normal z-score of study i \\ k = number of studies \end{array} \end{matrix}

Fail-safe N only addresses one specific failure mode (studies missing entirely) and says nothing about selective outcome or analysis reporting within studies that were published. It is normally reported alongside, not instead of, a funnel plot and Egger's regression test (which regresses each study's standardized effect on its precision, testing whether smaller, less precise studies systematically report larger effects — the visual asymmetry a funnel plot shows).

References

The formulas above follow the notation and derivations in Borenstein, Hedges, Higgins, & Rothstein (2009), Introduction to meta-analysis — see the full reference list for further reading on each method.