Multilevel models

The fixed-effect and random-effects models both assume one independent effect size per study. That assumption breaks when a study reports several effect sizes that share the same sample — several outcomes, several timepoints, or several subscales — because those effect sizes are statistically dependent, not independent draws. Ignoring the dependency understates the true variance of the pooled estimate: it silently double-counts information from the same participants, so confidence intervals come out too narrow and significance tests too liberal.

A multilevel (three-level) model, as formalized for meta-analysis by Van den Noortgate et al. (2013), adds a variance component for this extra layer of nesting:

Level 1 — sampling variance of each individual effect size ( $v_{i j}$ , for effect size j in study i), as in the models above.
Level 2 — variance between effect sizes within the same study ( $σ_{(2)}^{2}$ ): how much a study's own outcomes/timepoints vary around that study's true effect.
Level 3 — variance between studies ( $σ_{(3)}^{2}$ ): how much studies vary around the overall true effect.

\begin{matrix} {Var}_{(Y_{i j})} = v_{i j} + σ_{(2)}^{2} + σ_{(3)}^{2} \\ \begin{array}{l} Y_{i j} = effect size j from study i \\ v_{i j} = sampling variance of that effect size \\ σ_{(2)}^{2} = level-2 (within-study) variance \\ σ_{(3)}^{2} = level-3 (between-study) variance \end{array} \end{matrix}

Level 2 and level 3 together generalize the single τ² of the random-effects model into two components — one for within-study heterogeneity, one for between-study heterogeneity.

Three-level meta-analysis: level 3 variance between studies, level 2 variance between effect sizes nested within each study, level 1 sampling variance of each effect size

An alternative to fitting the full three-level model is robust variance estimation (RVE): fit a simpler working model (e.g., ordinary random-effects, or an assumed within-study correlation) and correct the standard errors afterward to account for the dependency, without needing to estimate the level-2/level-3 split explicitly. RVE is a common choice when the number of studies is small, since three-level models need enough studies to estimate all variance components reliably.

A widely used middle ground, sometimes called the correlated-effects working model, doesn't try to estimate the level-2 variance at all: it assumes a plausible within-study correlation (e.g., 0.5 — a common default when the true value isn't known) instead of estimating it, and only estimates the between-study variance τ². This is a reasonable simplification for two reasons. First, with the modest number of studies typical of most meta-analyses, there usually isn't enough information to reliably estimate two separate variance components at once — the extra parameter the full model asks for often isn't well identified by the data anyway. Second, this working model is normally paired with robust variance estimation for the final standard errors, which corrects for the model's covariance structure being wrong (including the assumed correlation) — so the assumption only has to be plausible, not exactly right, for the reported inference to stay valid. ProMeta implements this approach: a correlated-effects working model paired with cluster-robust (CR2) variance estimation.

Reach for a multilevel model (or RVE) whenever any study contributes more than one effect size to the same analysis, on top of whichever variance structure (fixed or random) applies at the study level.