LLM Temperature & Top-p Sampling Visualizer

A language model does not pick the next word directly. It outputs a raw score (a logit) for every token in its vocabulary. Sampling parameters turn those logits into a probability distribution and then decide how adventurously to draw from it. This tool applies the three parameters people actually search for, in the common reference order used by Hugging Face's generation code: temperature → top-k → top-p → renormalize.

1. Tempered softmax. Each logit is divided by the temperature T, then run through softmax: p_i = exp(z_i / T) / Σ_j exp(z_j / T). Lower T makes the top token more dominant (sharper, more confident); higher T flattens the distribution (more random). At T = 0 the math collapses to greedy decoding: the single highest token gets probability 1. We subtract the maximum scaled logit before exponentiating (the log-sum-exp trick) so large logits or tiny temperatures never overflow.
2. Top-k truncation. If k > 0, keep only the k highest-probability tokens and zero the rest (Fan et al., 2018). A fixed cutoff, regardless of how confident the model is.
3. Top-p (nucleus) truncation. If p < 1, sort the surviving tokens by probability and keep the smallest prefix whose cumulative probability reaches p (Holtzman et al., 2019). Unlike top-k, the cutoff adapts to the distribution's shape.
4. Renormalize. Divide the survivors by their sum so they total 1 again, these are the actual probabilities the sampler draws from. The badge in the calculator confirms they reconcile to 1.0.

The summary chips add an entropy readout, H = −Σ p·log₂ p in bits, as a single "how random is this" number, and an effective choices count, how many tokens still carry non-zero probability after truncation. The provider toggle clamps the slider ranges to each API's documented limits (OpenAI temperature 0–2 with no public top_k; Anthropic temperature 0–1 with optional top_k). Providers can differ in edge-case clamping and tie-breaking, so this tool states its reference order rather than claiming to mirror any one backend exactly.

Where the input logits come from

Real logits are model-specific and almost never exposed in full: an API returns the log-probabilities of a handful of top candidates, not the whole vocabulary. So this page works the other way round. You supply a short candidate list (2 to 12 tokens) with plausible logits, or load a preset shaped like a real case: a peaked factual distribution, a flat creative one, a code completion with one strong favourite, and a uniform coin flip. The math is identical whether the list holds four entries or 128,000, because softmax normalizes over whatever set it is handed. What a short list loses is the long tail, so read the truncation percentages as the shape of the effect rather than a literal prediction for a production model.

Three details that trip people up

• The crossing token is kept. Top-p keeps the smallest prefix whose cumulative probability is at least p, so the token that pushes the sum over the line stays in. In worked example B three tokens reach 0.9216 against a threshold of 0.8, and all three survive. The kept mass is therefore usually larger than p and never smaller, which is why p = 0.9 can behave far more aggressively than the number looks.
• Top-p is measured before renormalizing. With top-k and top-p both on, the cumulative sum runs over the temperature-scaled probabilities of the tokens that survived top-k, not over a set that was already rescaled. Renormalization happens once, at the very end. Swap that order and you get a different survivor list, which is one reason two libraries can disagree on identical settings.
• Equal logits are immune to temperature. A uniform distribution stays uniform at every T above 0, because dividing identical logits by the same number leaves them identical. The coin-flip preset shows it: 50/50 at T = 0.1 and 50/50 at T = 2. Temperature redistributes the gaps between logits, and where there is no gap there is nothing to redistribute.

Reading the two summary chips

Entropy answers "how much will this vary if I sample it a thousand times", and it has a ceiling: log₂(n) bits for n candidates, reached only when every token is equally likely. Six candidates cap out at 2.585 bits, which is why pushing temperature from 1.0 to 1.6 on the creative preset moves entropy only from 2.52 to 2.56. High temperature has sharply diminishing returns. Effective choices counts tokens that still carry non-zero final probability, so it falls when top-k or top-p truncates, and also when a probability underflows to zero in floating point (worked example E). For the sequence-level version of the same measure, the AI perplexity calculator works on whole-text log-probabilities instead of a single step.

What this tool does not model

One next-token step, three parameters. It deliberately leaves out everything that reshapes logits before or across steps: repetition and frequency penalties, which subtract from tokens already used (see the repetition penalty calculator), per-token logit_bias nudges (the logit bias calculator), the newer truncation families min-p and locally typical sampling (the min-p sampling calculator), beam search, and grammar-constrained decoding that masks whole token classes. It also stops after one step, so it says nothing about how a single unlucky draw compounds across a 500-token answer. That is the usual reason a high temperature reads as fine in a demo and drifts in a long generation.

Every figure below is recorded as a fixture in the calculator module, so the page and the tool cannot drift apart, and the final column always reconciles to 1.0 within 1e-4. Examples A to C use the default four-token list, D shows the two truncation filters interacting, and E and F are the edge cases that break most people's mental model of what these numbers do.

None of the ranges below come from a specification; they are what teams settle on, with the reasoning attached so you can disagree on purpose rather than by accident. Load the matching preset, set the value, and watch the entropy chip: that number, not the slider position, is what decides how much two runs of the same prompt will differ.

• 0 to 0.2, extraction and classification. You want the same label for the same input every time, and any variety is a defect. Entropy should read close to 0.
• 0 to 0.3, code and SQL. One wrong identifier breaks the file, and the second-best token is usually worse rather than merely different. Keep top_p at 1 so the temperature is the only thing changing.
• 0.3 to 0.7, summaries, support replies, rewriting. Enough movement to avoid stiff repeated phrasing, not enough to invent facts that were not in the source.
• 0.7 to 1.0, open chat. The default for most assistant APIs, and the range where the top token still wins most of the time while real alternatives stay reachable.
• 1.0 and above, brainstorming, names, titles. Worth it only when you will read several candidates and pick one. Watch the entropy ceiling: with six candidates it cannot pass 2.585 bits however high you push T.

Two cautions the sliders make obvious. First, a "mild" top_p is not mild on a confident distribution: worked example F turns p = 0.9 into plain greedy decoding, because one token already holds 97.9% of the mass. Top-p is a cutoff on cumulative probability, not a strength dial. Second, moving temperature and top_p at once makes the outcome hard to attribute, which is why OpenAI's own reference tells you to alter one or the other. Change one, note the entropy, then change the other.

Which parameters you can set at all depends on the provider: OpenAI exposes temperature 0 to 2 and top_p but no public top_k, Anthropic caps temperature at 1 and does expose top_k, and several newer reasoning models reject sampling parameters outright. The model parameter support checker lists what each model accepts. One thing sampling never changes is the bill: cost tracks tokens in and out, so a temperature change is free, and for that side of the arithmetic use the AI token counter.

• OpenAI API Reference: Chat Completions temperature & top_p
• Anthropic Messages API Reference: temperature, top_p, top_k
• Holtzman et al. (2019): The Curious Case of Neural Text Degeneration (top-p / nucleus sampling)
• Fan, Lewis & Dauphin (2018): Hierarchical Neural Story Generation (top-k sampling)
• Goodfellow, Bengio & Courville (2016): Deep Learning (tempered softmax)

Parameter ranges and semantics were last cross-checked against the OpenAI and Anthropic API references on 2026-06-11. The sampling math is standard; the worked examples above double as the tool's reconciliation fixtures (tolerance 1e-4).