What is a good chrF score?

chrF is reported on a 0–100 scale and runs noticeably higher than BLEU for the same system. As a rough guide, under 20 means little overlap, 40–50 is understandable mid-range MT, 50–65 is fluent and usable, and above 65 is close to the reference. There is no official threshold table, so only compare scores computed with identical settings and reference counts.

How is chrF different from BLEU?

BLEU counts matching word n-grams and multiplies a precision-only geometric mean by a brevity penalty. chrF counts matching character n-grams and takes an F-score, so it rewards recall directly and needs no brevity penalty. Because it works on characters, chrF is tokenisation-independent and handles morphology and word-order variation better, which is why WMT recommends it alongside BLEU.

Is chrF calculated on words or characters?

On characters. Both the candidate and reference have their whitespace removed, then every contiguous run of 1 to 6 characters is counted as an n-gram. 'the cat' becomes the six-character string 'thecat'. This is what makes chrF tokenisation-free and useful for morphologically rich or non-space-delimited languages such as Sinhala and Tamil.

What is chrF2 and what does the 2 mean?

The number is β, the recall weight in the F-score. chrF2 uses β = 2, so recall counts twice as much as precision, the standard configuration recommended by Popović (2016) and used by sacreBLEU. chrF1 (β = 1) weights precision and recall equally; chrF3 weights recall three times. Set β with the control above the results; the default here is 2.

Does chrF include spaces in the character n-grams?

Not in the sacreBLEU default this tool follows: all whitespace is stripped before the n-grams are extracted, so spaces never appear inside a character n-gram. Popović's original paper kept spaces as ordinary characters, which shifts scores slightly. This tool fixes the whitespace-stripped default to stay comparable with sacreBLEU; the methodology section documents the choice.

How do I score against multiple references?

Put one reference per line in the reference box. The tool computes chrF against each reference independently and reports the highest score, along with which reference matched best. This mirrors sacreBLEU's multi-reference behaviour: a candidate is credited for matching whichever human translation it is closest to.

Why is my chrF not exactly the same as sacreBLEU's number?

Check three things: β (this tool defaults to 2, chrF2), the character order (default 6), and lowercasing (off by default, sacreBLEU is case-sensitive). Also make sure you are comparing sentence-level to sentence-level; a corpus score aggregates counts differently. With matching settings the numbers reconcile, since both follow the Popović (2015) definition with whitespace stripped.

Does an identical translation always score 100?

Yes. When the candidate and a reference are character-for-character identical, every n-gram of every order matches, so chrP = chrR = 1 and chrF = 1.0 (100.0). Unlike METEOR, chrF gives a clean 100 for an exact match, which makes it easy to sanity-check the tool: paste the same text in both boxes and you should see 100.

AI · Machine Translation

chrF Score Calculator (Character n-gram F-score)

Paste a candidate translation and one or more references to get the chrF2 score on the 0–100 and 0–1 scales, the per-order character precision and recall, and the averaged chrP and chrR, so you can see exactly how the number was derived. Matches sacreBLEU, no signup, runs in your browser.

By Induwara Ashinsana— Co-Founder, Ryzera TechnologiesUpdated Jul 9, 2026

chrF Score

Candidate (machine output)

One sentence. Stays on your device.0 chars

Reference(s): one per line

At least one. Stays on your device.0 chars

Char order (N)β

Examples

chrF2

—

Rating

—

chrP · chrR

—

Best reference

—

Runs entirely in your browser, your text is never uploaded, logged, or stored. Method: character n-gram precision and recall averaged over the effective orders, combined with F-β, per Popović (2015); sacreBLEU default configuration (char order 6, β=2, whitespace stripped). Up to 50,000 characters per box.

How it works

chrF (character n-gram F-score), introduced by Maja Popović at WMT 2015, scores a machine-translation candidate against one or more reference translations using overlapping character n-grams rather than word n-grams. Working on characters makes it tokenisation-independent and gives partial credit for near-miss word forms, which is why the WMT shared tasks recommend it alongside BLEU. It is computed in six steps:

Preprocess. Optionally lowercase, then remove all whitespace from the candidate and each reference. Following the sacreBLEU default, the cat becomes the six-character string thecat.
Count character n-grams.For each order n from 1 to N (N = 6 by default), extract every contiguous run of n characters as a multiset. Hₙ is the total in the hypothesis, Rₙ the total in the reference.
Clip matches. For each distinct n-gram, credit min(count_in_hyp, count_in_ref). Summing gives Mₙ, the clipped match total for order n.
Per-order precision and recall.chrPₙ = Mₙ / Hₙ and chrRₙ = Mₙ / Rₙ.
Average over the effective orders.Only orders where both the hypothesis and reference have n-grams are counted, a 4-character hypothesis has no 5- or 6-grams, so those orders are skipped. With K counted orders, chrP and chrR are the arithmetic means of chrPₙ and chrRₙ over those K orders.
Combine with F-β. chrFβ = (1 + β²)·chrP·chrR / (β²·chrP + chrR). With β = 2 (the default, giving chrF2) recall is weighted twice as heavily as precision. Multiply by 100 for the conventional scale.

The F-β combination is computed directly and then re-derived through its reciprocal (harmonic) form as an independent cross-check; when the two agree the score is flagged “cross-checked”. The configuration, character order 6, β = 2, whitespace stripped, case-sensitive, is the sacreBLEU default described by Post (2018), so a sentence-level score here reconciles with sacrebleu.CHRF under matching settings. Because character order, β and case all change the number, report chrF together with its settings.

Worked examples

One letter off → 28.06

Candidate: the bat
Reference: the cat

Strip whitespace: hyp 'thebat', ref 'thecat' (6 chars each)
chrP₁..₆ = 5/6, 3/5, 1/4, 0, 0, 0 (chrR identical, equal length)
All six orders present in both → K = 6
chrP = 1.683333 / 6 = 0.280556; chrR = 0.280556
chrP = chrR, so chrF2 = chrP = 0.280556
chrF2 = 0.2806 → 28.06 / 100

Truncated hypothesis → 72.57

Candidate: hell
Reference: hello

Strip whitespace: hyp 'hell' (4), ref 'hello' (5)
chrP₁..₄ = 1, 1, 1, 1 (every hyp n-gram is in the reference)
chrR₁..₄ = 4/5, 3/4, 2/3, 1/2; orders 5–6 skipped (hyp too short)
K = 4 → chrP = 1.0, chrR = 2.716667 / 4 = 0.679167
chrF2 = 5·(1·0.679167) / (4·1 + 0.679167) = 3.395833 / 4.679167
chrF2 = 0.7257 → 72.57 / 100

Identical text → 100.00

Candidate: hello world
Reference: hello world

Strip whitespace: both become 'helloworld' (10 chars)
Every n-gram of every order 1–6 matches with equal count
chrP₁..₆ = 1 and chrR₁..₆ = 1 → chrP = chrR = 1.0
chrF2 = 5·1·1 / (4·1 + 1) = 5 / 5 = 1.0
chrF2 = 1.0000 → 100.00 / 100 (chrF is exact for identical text)

Frequently asked questions

Sources & references

The formulas and the worked examples on this page were last reconciled against Popović (2015) and the sacreBLEU chrF default on 2026-07-09. The calculation module ships with a built-in assertion that re-runs every worked example, so a regression in the chrF math fails fast.

Related tools

GLEU Score Calculator

Calculate the GLEU score (Google-BLEU, Wu et al. 2016) between a reference and a model hypothesis, the minimum of n-gram recall and precision over 1–4 grams. Shows the counted n-grams so the number is auditable. Matches NLTK sentence_gleu, runs in your browser, no signup.

Open tool

ROUGE Score Calculator

Calculate ROUGE-1, ROUGE-2, ROUGE-L and ROUGE-Lsum precision, recall and F1 between a generated summary and one or more references, entirely in your browser. Shows matched n-grams and the longest common subsequence. Matches Google rouge-score, no signup.

Open tool

Exact Match & F1 Calculator

Compute SQuAD-style Exact Match and token-level F1 between predicted and gold answers, single pair or batch, using the official SQuAD evaluation script's normalization and scoring rules. Runs entirely in your browser, no upload.

Open tool

Comments & feedback

Spotted a bug or want an improvement? Tell us, our team reviews every comment, and good ideas get built. Comments are public and anonymous.

Found a bug, edge case, or want to suggest an improvement?

Email me at [email protected] , most fixes ship within 24 hours.