Top-p vs Top-k Sampling Calculator

A language model produces a probability for every token in its vocabulary. Decoding parameters decide how much of that distribution to keep before drawing the next token. top_k and top_p are the two most common truncation knobs. This tool takes the probabilities as given — the post-temperature distribution, which is exactly the order real runtimes apply these filters (temperature → top_k → top_p) — and shows the cut in the reference order sort → top_k → top_p → renormalize.

1. Sort. Order the tokens by descending probability. Both filters operate on this sorted list.
2. top_k (Fan et al., 2018). If top_k = k > 0, keep only the k highest-probability tokens and zero the rest. k = 0 disables the filter, and a k larger than the vocabulary is a no-op.
3. top_p / nucleus (Holtzman et al., 2019). Walk the tokens in descending order, accumulating a cumulative sum. Include tokens until the cumulative sum first reaches or exceeds top_p — the token that crosses the threshold is included, then stop. top_p = 1.0 disables the filter; the walk uses the raw (not renormalized) masses, matching llama.cpp.
4. Renormalize. Let Sbe the sum of the kept tokens' original probabilities. Each kept token's sampling probability is p_i / S; cut tokens are 0. The survivors sum to exactly 1.

There is one insight worth internalising: because top_k and top_p both keep a prefix of the probability-sorted list, they commute. Combining them just keeps the shorter prefix — min(k, nucleus length) — so the two documented conventions (llama.cpp's “top_k then top_p” and the OpenAI-style “intersection”) always produce the same surviving set. The toggle in the tool changes only which filter is credited with cutting each token. As a credibility check, the nucleus length is computed two independent ways — an accumulate-and-break walk and an exclusive-prefix test — and the tool confirms they agree. Everything is deterministic and runs entirely in your browser.

All three use the peaked base distribution the=0.40, a=0.25, an=0.15, this=0.10, that=0.06, some=0.04 (which already sums to 1).

• Holtzman, Buys, Du, Forbes & Choi (2019) — The Curious Case of Neural Text Degeneration (top-p / nucleus sampling)
• Fan, Lewis & Dauphin (2018) — Hierarchical Neural Story Generation (top-k sampling)
• OpenAI API reference — Chat Completions top_p parameter (nucleus sampling)
• Anthropic Messages API — top_p and top_k parameters
• llama.cpp — sampling documentation (top_k applied, then top_p)

The formulas above were last cross-checked against these sources on 2026-07-11. The sampling math is standard; the worked examples double as the tool's reconciliation fixtures — every surviving set renormalizes to exactly 1.