chrF Score Calculator (Character n-gram F-score)

chrF (character n-gram F-score), introduced by Maja Popović at WMT 2015, scores a machine-translation candidate against one or more reference translations using overlapping character n-grams rather than word n-grams. Working on characters makes it tokenisation-independent and gives partial credit for near-miss word forms, which is why the WMT shared tasks recommend it alongside BLEU. It is computed in six steps:

1. Preprocess. Optionally lowercase, then remove all whitespace from the candidate and each reference. Following the sacreBLEU default, the cat becomes the six-character string thecat.
2. Count character n-grams.For each order n from 1 to N (N = 6 by default), extract every contiguous run of n characters as a multiset. Hₙ is the total in the hypothesis, Rₙ the total in the reference.
3. Clip matches. For each distinct n-gram, credit min(count_in_hyp, count_in_ref). Summing gives Mₙ, the clipped match total for order n.
4. Per-order precision and recall.chrPₙ = Mₙ / Hₙ and chrRₙ = Mₙ / Rₙ.
5. Average over the effective orders.Only orders where both the hypothesis and reference have n-grams are counted — a 4-character hypothesis has no 5- or 6-grams, so those orders are skipped. With K counted orders, chrP and chrR are the arithmetic means of chrPₙ and chrRₙ over those K orders.
6. Combine with F-β. chrFβ = (1 + β²)·chrP·chrR / (β²·chrP + chrR). With β = 2 (the default, giving chrF2) recall is weighted twice as heavily as precision. Multiply by 100 for the conventional scale.

The F-β combination is computed directly and then re-derived through its reciprocal (harmonic) form as an independent cross-check; when the two agree the score is flagged “cross-checked”. The configuration — character order 6, β = 2, whitespace stripped, case-sensitive — is the sacreBLEU default described by Post (2018), so a sentence-level score here reconciles with sacrebleu.CHRF under matching settings. Because character order, β and case all change the number, report chrF together with its settings.

• Popović, M. (2015) — chrF: character n-gram F-score for automatic MT evaluation (WMT15)
• Popović, M. (2016) — chrF deconstructed: β parameters and n-gram weights (WMT16)
• Post, M. (2018) — A Call for Clarity in Reporting BLEU Scores (sacreBLEU, WMT18)
• sacreBLEU — the reference implementation (sacrebleu.CHRF, char_order=6, beta=2)

The formulas and the worked examples on this page were last reconciled against Popović (2015) and the sacreBLEU chrF default on 2026-07-09. The calculation module ships with a built-in assertion that re-runs every worked example, so a regression in the chrF math fails fast.