AI Model Elo Rating Calculator

Elo is a paired-comparison rating system: it assigns every player or model a single number so that the gap between two ratings predicts the probability that one beats the other. It was designed by physicist Arpad Elo for chess in the 1960s and formalised in his 1978 book; the same math now ranks AI models on human-preference leaderboards such as LMSYS Chatbot Arena, which fit a Bradley–Terry / Elo model to millions of blind pairwise votes. This tool implements the three operations you actually need, each a deterministic closed-form calculation.

1. Expected score (win probability). For A against B, E_A = 1 / (1 + 10^((R_B − R_A) / 400)), and E_B = 1 − E_A. The constant 400 is the Elo scale: a 400-point gap means 10-to-1 odds (≈90.9%). E_A is a probability between 0 and 1, so a draw counts as half a point of “expected score.”
2. Rating update. After a game with actual score S_A (1 for a win, 0.5 for a draw, 0 for a loss), R'_A = R_A + K·(S_A − E_A) and the mirror for B. The K-factor sets how fast ratings move. Because E_A + E_B = 1 and S_A + S_B = 1, the update is exactly zero-sum — A gains precisely what B loses — which the tool verifies on every result.
3. Win-rate → Elo gap (Bradley–Terry inversion). Given an empirical decisive win rate p = wins / total (draws excluded), invert the logistic to recover the gap Δ = R_A − R_B: Δ = −400·log10(1/p − 1). This is the estimate a leaderboard reverse-engineers from observed battles. It needs 0 < p < 1; a 0% or 100% record implies an infinite gap, so the tool clamps those cases and asks for more games.

All logarithms are base-10 and all arithmetic is IEEE-754 double precision, computed client-side. To guard against coding slips the calculator computes the expected score two algebraically-independent ways (from the two ratings and from their gap) and confirms they agree, and it round-trips every win-rate gap back through the expected-score formula to check it reproduces the input probability. The formulas and their citations are pinned in the lib/data/ai-elo-rating-calculator.ts module header with a 2026-07-08 verification date.

• Elo rating system — expected-score formula and K-factor update rule (after Elo, 1978)
• Bradley & Terry (1952) — Rank Analysis of Incomplete Block Designs (Biometrika), the paired-comparison model
• Chiang et al. (2024) — Chatbot Arena: evaluating LLMs by human preference with an Elo / Bradley–Terry system

The math on this page follows the Elo (1978) logistic formula & Bradley–Terry (1952); the formulas were last cross-checked against these sources on 2026-07-08. All three worked examples above (64.0% / 36.0%; 1512 / 1388 zero-sum; +70.4 Elo round-tripping to 60.0%) are reproduced exactly by the calculator.