What is the difference between frequency penalty and presence penalty?

Both subtract from a repeated token's logit, but differently. presence_penalty is flat: it removes a fixed amount once, the moment a token has appeared at all, no matter how many times. frequency_penalty scales with the count: it removes penalty × count, so a token seen five times is hit five times harder than one seen once. Use presence to discourage reusing any prior token; use frequency to specifically crush the tokens that keep looping.

What does repetition penalty do in an LLM?

repetition_penalty (θ) is HuggingFace and llama.cpp's anti-repetition knob, from the CTRL paper. For any token already seen, it divides the logit by θ if the logit is positive and multiplies by θ if negative — pushing seen tokens toward zero influence either way. Unlike OpenAI's penalties it does not scale with the count: seen once or seen ten times, the same θ applies. The default is 1.2; 1.0 turns it off.

What is a good value for frequency penalty?

For most chat and writing, 0.1 to 0.5 trims obvious repetition without harming fluency. Push toward 0.8 to 1.0 only when the model loops badly on a phrase. Above roughly 1.0 the output can turn strange as common but necessary tokens get suppressed. The valid range is −2.0 to 2.0; negative values encourage repetition. Load a preset here and watch the repeated token's probability to find the point where it drops enough but not too far.

Does presence penalty stop repetition completely?

No. presence_penalty is applied once per token regardless of how many times it recurs, so a token that has already looped ten times gets the same single deduction as one seen once. That flat, count-blind edit nudges the model toward fresh vocabulary but does not escalate against a hard loop. For a runaway repeat, frequency_penalty (which grows with the count) or a lower temperature is the stronger lever.

How is repetition penalty calculated with the 1.2 default?

For each seen token, take its raw logit z. If z > 0, the adjusted logit is z ÷ 1.2; if z ≤ 0, it is z × 1.2. A logit of 3.0 becomes 2.5; a logit of −1.0 becomes −1.2. Then the usual softmax turns the adjusted logits into probabilities. Because it divides rather than subtracts, θ = 1.2 is a fairly gentle penalty — the step table on this page shows the exact before-and-after for your own numbers.

Can I use frequency_penalty and repetition_penalty at the same time?

Not within one provider. OpenAI's API exposes frequency_penalty and presence_penalty (the subtractive rule) but not repetition_penalty. HuggingFace, TGI, and llama.cpp expose repetition_penalty (the divisive rule). They are two different conventions for the same goal, so this tool makes you pick one at a time — matching how the real APIs behave. Switch the convention toggle to compare them on the identical token table.

What are logits and where do the counts come from?

A logit is the raw, pre-softmax score a model assigns each vocabulary token for the next position — higher means more likely. The count is how many times that token already appears in the text so far, which is what both penalty rules read. This tool is a teaching sandbox: you supply a small logit-and-count table so the math is transparent. It does not run a tokenizer or a live model.

Does this tool call an API or send my data anywhere?

No. Every calculation runs in your browser from the logits and counts you type or a preset — no API key, no cost, nothing leaves the page, and it keeps working offline after load. It visualizes a single next-token step to build intuition for the penalty math. For counting tokens in real text, use the AI Token Counter; for temperature and top-p, use the LLM Temperature Visualizer.

AI · Sampling

Frequency, Presence & Repetition Penalty Calculator

Enter a small logit-and-count table, drag the penalties, and watch repeated tokens collapse in real time. Shows OpenAI's subtractive frequency_penalty and presence_penalty next to HuggingFace/CTRL's divisive repetition_penalty — with the exact softmax math, token by token. No API key, no cost, runs in your browser.

By Induwara Ashinsana— Executive Director, Ryzera TechnologiesUpdated Jul 10, 2026

Penalty visualizer

Probabilities sum to 1.0 ✓

Penalty convention

frequency_penalty and presence_penalty subtract from the logit. Ranges −2 to 2. Used by the OpenAI Chat Completions API.

Token-table preset

Tokens (one label, logit, count per line)

2–12 tokens. Logit −20 to 20; count 0–50 (prior occurrences). Illustrative scores, not a real model run.

frequency_penalty

Scales with each token's count

presence_penalty

Flat, applied once per seen token

repetition_penalty (θ)

HuggingFace / CTRL-only knob

temperature (T)

Neutral

Most likely after

today

52.18% · rank flipped

Mass on repeated tokens

47.82%

was 92.02% before

Distribution shift (TVD)

0.59

Reshaped

frequency_penalty removes 3.00 from a token seen 6× (0.5 × 6); presence_penalty removes a flat 0.50 from any token seen at least once.

Probability: before → after

thank×6

43.68%→8.62%

you×6

29.28%→5.78%

help×2

13.16%→19.2%

today

7.98%→52.18%

great×1

5.91%→14.22%

Step-by-step math

Token	Logit	Count	Δ logit	Adj. logit	Before	After
thank	3.2	6	-3.5	-0.3	43.68%	8.62%
you	2.8	6	-3.5	-0.7	29.28%	5.78%
help	2	2	-1.5	0.5	13.16%	19.2%
today	1.5	0	0	1.5	7.98%	52.18%
great	1.2	1	-1	0.2	5.91%	14.22%

Copy-ready request body

{
  "frequency_penalty": 0.5,
  "presence_penalty": 0.5
}

One convention active at a time. Sources: .

How it works

A language model scores every vocabulary token with a raw logit before it picks the next word. The three anti-repetition parameters — frequency_penalty, presence_penalty, and repetition_penalty — all work by editing those logits based on how often each token has already appeared, then letting the usual softmax turn the edited logits into probabilities. They come in two different, non-interchangeable conventions, and this tool runs exactly one at a time, just like the real APIs.

OpenAI — subtractive. Per OpenAI's text-generation guide, for token j with count c[j]:

mu[j] = mu[j] − c[j] × frequency_penalty − (c[j] > 0 ? 1 : 0) × presence_penalty

The frequency term scales with the count — a token seen five times loses five times as much — while the presence term is a flat, once-off deduction the instant a token has appeared at all. Both range −2.0 to 2.0; negatives encourage repetition.

HuggingFace / CTRL — divisive. Introduced by Keskar et al. (2019) and implemented by the HuggingFace RepetitionPenaltyLogitsProcessor, for any seentoken (count > 0):

adj[j] = logit[j] / θ if logit[j] > 0, else logit[j] × θ

Dividing a positive logit or multiplying a negative one both push the token toward zero influence. θ (the repetition penalty) runs 1.0 to 2.0, defaults to 1.2, and — unlike the OpenAI rule — does not scale with the count: seen once or seen ten times, the same θ applies.

Then the tempered softmax. Whichever convention is active, the adjusted logits are turned into probabilities with p_j = exp(adj[j] / T) / Σ_k exp(adj[k] / T), where T is the temperature. The tool subtracts the maximum scaled logit before exponentiating (the log-sum-exp trick), so extreme logits or small temperatures never overflow. The badge in the calculator confirms the output probabilities reconcile to 1.0, and a second, independent softmax routine cross-checks the first to within 1e-4.

Worked examples

1 · OpenAI subtractive — the "thank you" repeat

A: thank, logit 3.0, count 4 · B: instead, logit 2.0, count 0 · freq 0.5, presence 0.5, T 1

adj[A] = 3.0 − 0.5×4 − 0.5×1 = 0.5
adj[B] = 2.0 − 0.5×0 − 0.5×0 = 2.0
Before: p[A] = e³/(e³+e²) = 73.11%, p[B] = 26.89%
After: p[A] = e^0.5/(e^0.5+e²) = 18.24%, p[B] = 81.76%
Repeated A collapses 73.11% → 18.24%; fresh B now leads

2 · HuggingFace / CTRL divisive — θ = 1.2

Same tokens · penalties off, θ 1.2, T 1

adj[A] = 3.0 ÷ 1.2 = 2.5 (positive logit, seen → divide)
adj[B] = 2.0 (count 0 → unchanged)
Before: p[A] = 73.11%, p[B] = 26.89%
After: p[A] = e^2.5/(e^2.5+e²) = 62.25%, p[B] = 37.75%
Milder than subtractive (62.25% vs 18.24%) — the conventions differ

3 · Edge case — frequency scales, presence does not

A: logit 2.0, count 1 · B: logit 2.0, count 5 · freq 0.3, presence 0, T 1

adj[A] = 2.0 − 0.3×1 = 1.7; adj[B] = 2.0 − 0.3×5 = 0.5
Before: equal logits → 50.00% each
After: p[A] = e^1.7/(e^1.7+e^0.5) = 76.85%, p[B] = 23.15%
The more-repeated B drops to 23.15%; presence-only would keep both at 50%

Frequently asked questions

Sources & references

Penalty ranges and the two editing rules were last cross-checked against the OpenAI API reference, the CTRL paper, and the HuggingFace processor on 2026-07-10. The worked examples above double as the tool's reconciliation fixtures (tolerance 1e-4).

Related tools

LiveAI

Presence & Frequency Penalty

See exactly how OpenAI's presence_penalty and frequency_penalty reshape next-token logits and softmax probabilities. Enter candidate tokens with logits and prior counts, drag the sliders, and watch the most likely next token flip. Uses the exact additive formula, runs in your browser.

Open tool

LiveAI

Top-p & Top-k Calc

Interactive calculator for LLM top-k and top-p (nucleus) sampling. Drag top_k or top_p on a next-token distribution and watch which candidates survive, get zeroed, and renormalize, with the exact math token by token. Runs in your browser, no API key, sources cited.

Open tool

LiveAI

AI Min-p Sampling Calc

Interactive calculator for LLM min-p and locally-typical sampling. Drag min_p or typical_p and watch which next-token candidates survive, get zeroed, and renormalize, with the exact math token by token. Runs in your browser, no API key, sources cited.

Open tool

Rate this tool

Be the first to rate

Comments & feedback

Spotted a bug or want an improvement? Tell us — our team reviews every comment, and good ideas get built. Comments are public and anonymous.

Found a bug, edge case, or want to suggest an improvement?

Email me at [email protected] — most fixes ship within 24 hours.