GLEU Score Calculator (Google-BLEU)

GLEU — Google-BLEU — is a sentence-level metric for machine translation and grammatical error correction. It was defined by Wu et al. (2016) for Google's Neural Machine Translation system, where corpus-level BLEU was a poor per-sentence reward. GLEU compares a model's output (the hypothesis) with a correct reference and rewards shared n-grams from both directions at once. The definition is:

GLEU = min( recall, precision )

where recall and precision are built from the same matching-n-gram count. The steps are:

1. Tokenise.The reference and hypothesis are optionally lowercased and split on whitespace. By default, leading and trailing punctuation is split into its own token so that “end.” becomes two tokens.
2. Record n-grams. For every order n from 1 to N (N = 4 in the GNMT paper), list the multiset of n-grams in the reference and in the hypothesis.
3. Count matches with clipping. The matching count M is the size of the multiset intersection — M = Σ min(count_hyp(g), count_ref(g)) over every n-gram g and every order. Clipping stops a repeated correct word from being credited more times than it truly appears.
4. Divide and take the minimum. With total_R reference n-grams and total_H hypothesis n-grams, recall = M / total_R and precision = M / total_H. GLEU is the smaller of the two.

Unlike BLEU, GLEU applies no brevity penalty. It does not need one: a hypothesis that is too short misses reference n-grams and loses recall, while one that over-generates inflates total_H and loses precision. The final min()is bound by whichever failure is worse, which is exactly why GLEU is stable on single sentences where BLEU's geometric mean can collapse to zero. Orders longer than a sentence simply contribute zero n-grams on that side, so short inputs stay division-safe. This tool re-counts the intersection from both the hypothesis and the reference side and confirms the two agree, and it reconciles with NLTK's sentence_gleu for a single reference under matching tokenisation.

• Wu et al. (2016) — Google's Neural Machine Translation System (arXiv:1609.08144), the GNMT / Google-BLEU (GLEU) definition
• NLTK nltk.translate.gleu_score — reference implementation (sentence_gleu, corpus_gleu)
• Napoles, Sakaguchi, Post & Tetreault (2015) — Ground Truth for Grammatical Error Correction Metrics (the distinct source-aware GEC-GLEU)
• Papineni, Roukos, Ward & Zhu (2002) — BLEU (the clipped n-gram counting convention GLEU shares)

The formula and the worked examples on this page were last reconciled against Wu et al. (2016) on 2026-07-10. The calculation module ships with a built-in assertion that re-runs every worked example, so a regression in the GLEU math fails fast.