Exact Match & Token F1 Score Calculator (SQuAD)

Exact Match and token-level F1 are the two standard metrics for extractive question answering, introduced with the Stanford Question Answering Dataset (SQuAD). This tool is a faithful port of the official SQuAD evaluate-v2.0.py — the same normalize_answer, compute_exact and compute_f1 functions researchers cite — so the numbers match what you would get from the Python evaluate library.

Every answer is first normalized so scoring rewards finding the right span, not copying grammar. Normalization applies four steps in order:

1. Lowercase the text.
2. Remove punctuation — every character in the ASCII punctuation set !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~.
3. Remove the articles a, an, and the as standalone words (regex \b(a|an|the)\b).
4. Collapse runs of whitespace into single spaces.

The two metrics are then computed on the normalized strings and their whitespace tokens:

• Exact Match — EM = (normGold === normPred) ? 1 : 0. A single differing character drops it to 0.
• Token F1 — with numSame the multiset overlap of gold and predicted tokens, precision = numSame / |pred|, recall = numSame / |gold|, and F1 = 2·precision·recall / (precision + recall). The tool also re-derives F1 as the Dice form 2·numSame / (|pred| + |gold|) — algebraically identical — and shows both, so you can see the score computed two independent ways.

Two edge cases follow the official script exactly. First, the no-answer rule from SQuAD 2.0: if either side has no tokens, F1 is 1 only when both are empty (they agree that there is no answer) and 0 otherwise. Second, multiple gold answers: the prediction is scored against each acceptable answer and the maximum is kept, independently for EM and F1. For a dataset, EM and F1 are macro-averaged across rows and reported as percentages.

• Rajpurkar, Zhang, Lopyrev & Liang — SQuAD: 100,000+ Questions for Machine Comprehension of Text (EMNLP 2016)
• Official SQuAD evaluation script — evaluate-v2.0.py (rajpurkar/SQuAD-explorer)
• Rajpurkar, Jia & Liang — Know What You Don't Know: Unanswerable Questions for SQuAD (ACL 2018)

The normalization, EM and F1 functions were last cross-checked against the Rajpurkar et al., EMNLP 2016 (arXiv:1606.05250) and the official SQuAD evaluate-v2.0.py on 2026-07-12. All four worked examples above are reproduced exactly by the calculator.