Cohen's Kappa Calculator

A free calculator for inter-rater agreement, in your browser: enter how two raters classified the same items and get Cohen's kappa, the observed and the expected agreement, and a verbal interpretation. It is an area of ProMeta. Nothing is saved or sent anywhere.

Open the calculator

What it does

• Generic calculator: an agreement matrix from 2 to 10 categories.
• Three weightings: unweighted nominal kappa (categories without order, every disagreement counts the same), linear weighted kappa (ordered categories, near misses penalized less) and quadratic weighted kappa (ordered categories, large disagreements penalized more).
• Study-selection preset: a fixed 2x2 Inclusion/Exclusion matrix for the check usually reported in a systematic review, where two reviewers independently decide whether each record is included. Unweighted, interpreted with the original benchmark labels of Landis and Koch (1977).
• Output: kappa, observed agreement (P_o), expected agreement by chance (P_e) and the interpretation.

When to use it in a meta-analysis

Agreement between two reviewers is usually reported twice: when selecting studies (include or exclude) and when coding them (for example the risk-of-bias judgment or a categorical moderator). See Selecting primary studies and Coding primary studies.

Interpretation

Landis and Koch (1977) label kappa as slight (0.01–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80) and almost perfect (0.81–1.00) agreement. A value of 1 is perfect agreement; negative values mean agreement below what chance alone would produce. These labels are a convention, not a threshold of quality: report the value itself.

Shares ProMeta's releases, known bugs and privacy policy. Runs entirely in the browser (see Data & Security).