What is a good intraclass correlation coefficient value?

Following Koo & Li (2016), values below 0.50 indicate poor reliability, 0.50–0.75 moderate, 0.75–0.90 good, and above 0.90 excellent reliability. Most journals expect at least 0.75 (good) before treating raters or repeat measurements as interchangeable. The threshold you need depends on how the measurement will be used.

What is the difference between ICC(2,1) and ICC(3,1)?

Both use a two-way model, but ICC(2,1) is absolute agreement (it penalises systematic rater differences) while ICC(3,1) is consistency (it ignores them). Report ICC(2,1) when the actual score matters, e.g. two clinics must produce the same number. Report ICC(3,1) when only the ranking or trend matters. ICC(3,1) is always ≥ ICC(2,1).

Should I use absolute agreement or consistency for ICC?

Use absolute agreement (form 2) when raters must give the same actual value and you will treat their scores as interchangeable, the usual choice for inter-rater reliability. Use consistency (form 3) when a fixed set of raters will always score every subject and you only care that they rank subjects the same way. When in doubt, absolute agreement is the safer, more conservative report.

What is the difference between ICC and Cohen's kappa?

ICC is for continuous or ordinal measurements (angles, scores, timings), while Cohen's kappa is for categorical ratings (yes/no, category labels). If your data are numbers on a scale, use ICC; if they are categories, use kappa. For nominal data with three or more raters use Fleiss' kappa, and for mixed data types use Krippendorff's alpha.

Can ICC be negative, and what does a negative ICC mean?

Yes. A negative ICC means the within-subject variation exceeds the between-subject variation, raters disagree more than you would expect by chance. It signals systematic disagreement, not agreement, and is usually reported as 0 reliability. This calculator shows the true negative value rather than clamping it, so you can see the problem.

How many subjects and raters do I need for a reliable ICC?

ICC needs at least 2 subjects and 2 raters to compute, but a stable estimate needs more. Koo & Li (2016) recommend at least 30 subjects and 3 raters for a trustworthy reliability study. Small samples give wide confidence intervals, so treat an ICC from 5–10 subjects as indicative rather than definitive.

Does this ICC calculator handle missing data?

No. The classic Shrout & Fleiss ICC is defined only for a complete grid where every subject is rated by every rater. If a cell is blank, remove that subject or use the Krippendorff's alpha tool, which is designed for incomplete data. This calculator rejects ragged or incomplete grids with a clear message rather than guessing.

Why do you not show a confidence interval or p-value?

This version reports the ICC point estimate and reliability verdict, which is what most searchers need to pick and cite the correct form. Exact F-based confidence intervals (McGraw & Wong, 1996) are planned for a later version. For CIs today, run the same data through R's psych::ICC or irr::icc, the point estimates will match this tool.

Statistics · Reliability

Intraclass Correlation Coefficient (ICC) Calculator

Paste a subjects × raters grid and get all six ICC forms at once, with the ANOVA table, a plain-language reliability verdict, and a methods sentence you can drop straight into a thesis or paper. Free, no signup, runs entirely in your browser.

By Induwara Ashinsana— Co-Founder, Ryzera TechnologiesUpdated Jul 10, 2026

ICC Calculator

Cross-verified

Ratings grid, one subject per row, one rater per column

6 subjects × 4 raters · grand mean 5.29

Load sample

Model

Agreement

Unit

Report this

ICC(2,1) = 0.290

Two-way random effects, single rater, absolute agreement

Poor

Below 0.50, poor reliability. Readings should not be pooled or treated as interchangeable.

Ready-to-paste methods line

Inter-rater reliability was poor, ICC(2,1) = 0.290 (two-way random-effects model, absolute agreement, single rater; Koo & Li, 2016).

All six ICC forms

Form	Model	ICC	Verdict
ICC(1,1)	One-way, single	0.166	Poor
ICC(2,1)	Two-way random, single (absolute)	0.290	Poor
ICC(3,1)	Two-way mixed, single (consistency)	0.715	Moderate
ICC(1,k)	One-way, average	0.443	Poor
ICC(2,k)	Two-way random, average (absolute)	0.620	Moderate
ICC(3,k)	Two-way mixed, average (consistency)	0.909	Excellent

Consistency forms (3,·) are always ≥ their absolute-agreement counterparts (2,·), and average-measure forms (·,k) always ≥ single-rater forms (·,1), that ordering is a quick sanity check on any ICC table.

ANOVA table

Source	SS	df	Mean square
Between subjects BMS	56.21	5	11.24
Between raters JMS	97.46	3	32.49
Residual / error EMS	15.29	15	1.02
Within subjects (one-way) WMS	112.75	18	6.26

Formulas: Shrout & Fleiss (1979) numbering, McGraw & Wong (1996) mean-square form. Verdict bands: Koo & Li (2016). All computation runs in your browser, no data is uploaded. Confidence intervals and F-test p-values are out of scope for this version; for those use R's psych::ICC or irr::icc.

What this tool does

The intraclass correlation coefficient (ICC) measures how much of the total variation in a set of measurements comes from real differences between subjects rather than from disagreement between raters or repeat measurements. It is the standard way to report inter-rater and test–retest reliability for continuous data , goniometer angles, pain scores, questionnaire totals, timing measurements, and it is what journal reviewers ask for. This tool computes it in your browser and never uploads your data.

How it works

Given a complete matrix of n subjects (rows) by k raters (columns), the calculator runs a two-way analysis of variance without replication, following McGraw & Wong (1996). It first computes four sums of squares from the grand mean, the per-subject (row) means, and the per-rater (column) means:

SS_R, between subjects: k · Σ(x̄ᵢ − x̄)²
SS_C, between raters: n · Σ(x̄ⱼ − x̄)²
SS_E, residual: SS_T − SS_R − SS_C
SS_W, within subject (one-way): SS_T − SS_R

Dividing each by its degrees of freedom gives the four mean squares, BMS, JMS, EMS and WMS, that every ICC formula is built from. The six Shrout & Fleiss (1979) forms are then:

ICC(1,1) = (BMS − WMS) / (BMS + (k−1)·WMS)
ICC(2,1) = (BMS − EMS) / (BMS + (k−1)·EMS + (k/n)·(JMS − EMS))
ICC(3,1) = (BMS − EMS) / (BMS + (k−1)·EMS)
ICC(1,k) = (BMS − WMS) / BMS
ICC(2,k) = (BMS − EMS) / (BMS + (JMS − EMS)/n)
ICC(3,k) = (BMS − EMS) / BMS

The numbers in the first slot are the model (1 = one-way random, 2 = two-way random, 3 = two-way mixed) and the second slot is the unit(1 = a single rater, k = the average of all raters). Form 2 measures absolute agreement. It counts systematic rater bias as disagreement, while form 3 measures consistency and ignores that bias, which is why ICC(3,·) is always at least as large as ICC(2,·). To guard against arithmetic slips, the tool recomputes every form a second way, from independent variance components (McGraw & Wong, 1996, Table 1), and only shows the “cross-verified” badge when both methods agree to six decimals. Reliability bands (poor / moderate / good / excellent) follow Koo & Li (2016). If every subject has the same average rating there is no between-subject variance and the ICC is reported as undefined rather than a misleading zero.

Worked examples

Example 1, Shrout & Fleiss (1979) 6 targets × 4 judges

Grid rows: 9 2 5 8 / 6 1 3 2 / 8 4 6 8 / 7 1 2 6 / 10 5 6 9 / 6 2 4 7
ANOVA: BMS = 11.2417, JMS = 32.4861, EMS = 1.0194, WMS = 6.2639
ICC(1,1) = (11.2417 − 6.2639) / (11.2417 + 3×6.2639) = 0.166
ICC(2,1) = 10.2222 / (11.2417 + 3×1.0194 + (4/6)(32.4861 − 1.0194)) = 0.290
ICC(3,1) = 10.2222 / (11.2417 + 3×1.0194) = 0.715
Averages: ICC(1,4) = 0.443, ICC(2,4) = 0.620, ICC(3,4) = 0.909
Matches the paper's Table 2 exactly (0.17 / 0.29 / 0.71 / 0.44 / 0.62 / 0.91).

Example 2, 3 subjects × 2 raters, hand-checkable

Grid: 4 5 / 7 8 / 9 9 (grand mean 7)
SS_T = 22, SS_R = 21, SS_C = 0.6667, SS_E = 0.3333
BMS = 10.5, JMS = 0.6667, EMS = 0.16667
ICC(3,1) consistency = 10.3333 / 10.6667 = 0.969 → Excellent
ICC(2,1) absolute = 10.3333 / 11.0 = 0.939 → Excellent
Consistency > absolute because rater 2 scores ~1 point higher on two subjects, a systematic bias that only absolute agreement penalises.

Example 3, degenerate grid (edge case)

Grid: 5 5 / 5 5 / 5 5 — every rating identical
Between-subject sum of squares SS_R = 0, so BMS = 0
Every ICC denominator collapses to 0
Result: all six forms are reported as Undefined, not 0 or NaN
The tool explains that there is no between-subject variance to model.

Frequently asked questions

Sources & references

Formulas and interpretation thresholds on this page were last cross-checked against these sources on 2026-07-10. The Shrout & Fleiss 6×4 dataset is used as the tool's regression test, the calculator reproduces every value in their Table 2.

Related tools

Education

Cronbach's Alpha Calculator

Compute Cronbach's alpha internal-consistency reliability from a respondents × items score grid, with per-item mean, variance, corrected item-total correlation, and alpha if item deleted. Runs entirely in your browser, no upload.

Open tool

Education

Wilcoxon Signed-Rank Test

Run the Wilcoxon signed-rank test on paired samples (before/after) or one sample against a median. Returns W⁺, W⁻ and W = min(W⁺, W⁻), the exact or tie-corrected normal-approximation p-value, the rank-biserial effect size, and the full ranking worktable. Runs in your browser, no signup.

Open tool

Education

Kruskal-Wallis Test

Run the Kruskal-Wallis H test online on 2+ groups: pooled ranks, tie-corrected H statistic, degrees of freedom, and a chi-square p-value, in-browser with every step shown. The non-parametric alternative to one-way ANOVA and the 3+-group extension of the Mann-Whitney U test.

Open tool

Comments & feedback

Spotted a bug or want an improvement? Tell us, our team reviews every comment, and good ideas get built. Comments are public and anonymous.

Found a bug, edge case, or want confidence intervals added?

Email me at [email protected] , most fixes ship within 24 hours.