Softmax Calculator (with Temperature)

The softmax function converts a vector of real-valued scores — called logits— into a probability distribution: every output is positive and the outputs sum to 1. It is the last layer of almost every classification neural network, and the function behind the temperature knob in modern language-model sampling. This calculator implements the standard, numerically-stable definition from Goodfellow, Bengio & Courville's Deep Learning (MIT Press, 2016).

For logits z = [z₁ … zₙ] and a temperature T, it runs these steps:

1. Temperature scaling. Each logit is divided by the temperature: sᵢ = zᵢ / T. At T = 1 this is the plain softmax (Hinton, Vinyals & Dean, 2015).
2. Stability shift. Subtract the maximum scaled score, dᵢ = sᵢ − max(s). This stops exp() from overflowing on large inputs. The constant cancels in the next division, so the answer is unchanged (Deep Learning §4.1).
3. Exponentiate. eᵢ = exp(dᵢ). Because every shifted score is ≤ 0, each exponential lands in (0, 1].
4. Normalise. pᵢ = eᵢ / Σⱼ eⱼ. This is the softmax probability for class i (Deep Learning §6.2.2; PyTorch softmax reference).

The predicted class is the argmax — the index with the highest probability, which is temperature-invariant because dividing by a positive T never reorders the scores. For a sharpness readout the tool also computes the Shannon entropy H = −Σ pᵢ log₂ pᵢ in bits, whose maximum of log₂ n marks a perfectly uniform distribution. To prove correctness the module cross-checks the stable result against the naive (no-shift) softmax: on well-conditioned inputs the two agree to roughly 1e-15, and on extreme inputs only the stable path survives — which is exactly why the shift exists. Everything is plain double-precision arithmetic; nothing is sent to a server.

What temperature actually changes

Temperature is the single most misunderstood part of the function, so it is worth being precise. Dividing by T does not add or remove information — it only changes how far apart the scores are before they are exponentiated. Because exp is monotonic and T is strictly positive, the ranking of the classes is untouched. What changes is the ratio between neighbouring probabilities: for any two classes i and j, that ratio is exactly pᵢ / pⱼ = exp((zᵢ − zⱼ) / T). A gap of 1.0 between two logits becomes a 2.72× odds ratio at T = 1, shrinks to 1.65× at T = 2, and blows out to 7.39× at T = 0.5.

That single expression explains both limits. As T grows without bound the exponent tends to 0, every ratio tends to 1, and the distribution converges on uniform — maximum entropy, maximum randomness. As T approaches 0 the exponent for the leading class tends to infinity, so all the probability mass collapses onto the argmax; the softmax degenerates into a hard, one-hot argmax. This is why greedy decoding is often described as “temperature 0”: it is the limiting case, not a separate algorithm. This calculator clamps T to the 0.1–5.0 range because T = 0 is a division by zero, and beyond about 5 the distribution is visually indistinguishable from uniform.

Edge cases and how they are handled

A calculator is only trustworthy if it is explicit about the awkward inputs, so here is what happens at each boundary:

• Negative logits. Perfectly valid. Softmax is shift-invariant, so [−1, −2, −3] produces exactly the same distribution as [0, −1, −2]. Only the differences between logits matter, never their absolute level.
• All-equal logits. [0, 0, 0] gives three probabilities of 0.3333 and entropy of log₂3 = 1.585 bits — the maximum. On an exact tie the argmax uses a strict greater-than comparison, so the earliest index is reported. Deterministic, never random.
• A single value. One logit always yields a probability of exactly 1.0000, whatever the number is, with entropy 0. That is the correct degenerate answer, not a bug.
• Huge magnitudes. Inputs like 1e9 overflow the naive formula to Infinity and then to NaN. The stability shift keeps the largest exponent at exp(0) = 1, so the result stays finite.
• Very negative shifted scores. Anything below roughly −745 underflows to a true 0 in double precision. That class then reads 0.0000, which is the intended floating-point answer — softmax is positive in exact arithmetic, but not at 64-bit resolution.
• Bad input. Non-numeric tokens are rejected by name rather than silently becoming NaN, and the vector is capped at 100 values to keep the render instant.

The most common question about softmax is why it bothers with exp when dividing each score by the sum of the scores would also produce numbers that add to 1. Run the canonical vector through that simpler idea and it looks fine: [2.0, 1.0, 0.1] has a sum of 3.1, so plain normalisation gives [0.6452, 0.3226, 0.0323] — close enough to softmax's [0.6590, 0.2424, 0.0986] that you might not notice the difference.

It falls apart the moment a logit is negative, and logits are negative all the time. Take [1, −2]: the sum is −1, so plain normalisation returns [−1, 2] — a negative probability and one above 100%. Take [1, −1]: the sum is exactly 0 and the whole thing is a division by zero. Neither input is exotic; both are ordinary outputs of a linear layer. Softmax has no such failure mode, because exp maps every real number, however negative, to a strictly positive one before anything is divided. The denominator is therefore always positive and the outputs are always in (0, 1).

There is a deeper reason too. Exponentiating treats the logits as log-odds rather than as raw proportions, which is what makes the gaps between them behave multiplicatively — the pᵢ / pⱼ = exp(zᵢ − zⱼ) identity from the previous section. Adding 1 to a logit multiplies that class's odds by e ≈ 2.72 regardless of where the scores started, so the function is shift-invariant and the network never has to learn a particular absolute scale. Plain normalisation has neither property: adding a constant to every score changes the answer, which means the layer feeding it would have to pin down an arbitrary offset.

The alternatives, and when they are used

Softmax is not the only way to turn scores into a decision, and it helps to know what it is competing with:

• Hard argmax (one-hot). Put all the mass on the winner and zero everywhere else. It is what you want at inference when you only need a label, and it is what a one-hot encoder produces for training targets. It cannot be used as a training output because its gradient is zero almost everywhere — there is nothing for backpropagation to descend. Softmax is the smooth stand-in, which is why the name is “soft” max.
• Sigmoid, per label. The right choice when labels are independent rather than mutually exclusive. Compare the two shapes side by side with the activation function calculator, which covers sigmoid, tanh, ReLU and GELU on the same axes.
• Sparsemax and entmax. Variants that can assign an exact zero to weak classes instead of a tiny positive number, while staying differentiable. Useful when you want a genuinely sparse attention or output distribution; rarely seen in general-purpose classifiers.
• Gumbel-softmax. Adds calibrated noise so you can sample a category and still backpropagate through the choice. This is the trick behind discrete latent variables; it reduces to ordinary softmax as its own temperature parameter falls.

Softmax is almost never the last thing you compute. During training its output feeds straight into a loss function, and at evaluation time it feeds a stack of metrics — which is why the numbers on this page tend to be an intermediate step rather than an answer on their own.

The immediate next step in training is the loss. Categorical cross-entropy takes the softmax probability assigned to the correct class and computes −log(p), so a confident correct prediction costs almost nothing and a confident wrong one costs a great deal. If you want to see that half of the calculation with the same step-by-step treatment, the cross-entropy loss calculator takes the probabilities this page produces and turns them into a loss value. In practice frameworks fuse the two steps into a single log-softmax-plus-negative-log-likelihood operation for numerical reasons, but the arithmetic is identical to doing it in two passes.

For language models the same per-token probabilities are aggregated across a whole sequence into perplexity, the standard readout of how surprised a model is by text it did not write — the perplexity calculator covers that conversion. When you need to compare two distributions rather than score one — a distilled student against its teacher, for instance, which is the original setting for temperature scaling — the measure is relative entropy, and the KL divergence calculator works on exactly the kind of probability vectors this tool emits. Once predictions are turned into hard class labels by taking the argmax, accuracy, precision and recall come from the confusion matrix calculator.

One caution worth stating plainly: a softmax probability is not a calibrated confidence. Modern deep networks are widely documented as overconfident — a model can report 0.99 on inputs it gets wrong, because nothing in the training objective forces the reported number to match the empirical hit rate. Softmax guarantees a valid probability distribution. It does not guarantee an honest one. Treat the output as a ranking with a magnitude attached, and check calibration separately before acting on the number.

The version of softmax most people meet today is not the three-class textbook one — it is the enormous one at the end of a transformer. After the final layer the model holds one logit per vocabulary entry, typically somewhere between 32,000 and 200,000 of them, and a single softmax over that whole vector produces the next-token distribution. Everything a sampler does afterwards is a modification of those probabilities. The logits themselves come out of the attention and feed-forward stack; if you want to see where the numbers upstream of this step originate, the attention score calculator walks through the scaled dot-product softmax that runs inside every attention head — the same function, applied to a different vector.

The table below is the canonical vector [2.0, 1.0, 0.1] run through this calculator at five temperatures. It is the clearest way to see what the knob actually does: the winner is identical in every row, while the odds ratio between the top two classes moves across four orders of magnitude.

Note how uneven the scale is. Dropping from T = 1 to T = 0.5 roughly triples the odds ratio, but dropping from 0.5 to 0.1 multiplies it by another three thousand — at T = 0.1 the distribution has already collapsed onto a single token in all but name. That asymmetry is why the useful range for text generation is narrow and why the difference between 0.7 and 0.9 matters far more than the difference between 3 and 5. Entropy makes the same point in one number, and the Shannon entropy calculator computes it for any distribution you already have, softmax or not.

Order of operations with top-k, top-p and min-p

Temperature is not the only sampler in the chain, and the order matters more than most documentation admits. The common pipeline is: compute logits, apply any logit bias, divide by temperature, softmax, then truncate with top-k, top-p (nucleus) or min-p, then renormalise the survivors and draw. Because truncation happens after the softmax, temperature indirectly controls how many tokens survive a top-p cut — a low temperature pushes so much mass onto the leading token that a nucleus of p = 0.9 may contain a single candidate, making top-p a no-op. Raise the temperature and the same p = 0.9 can admit dozens. The two settings are not independent, which is why tuning them one at a time tends to disappoint. The top-p and top-k sampling calculator shows the truncation step on a real distribution, the min-p sampling calculator covers the newer relative-threshold variant, and the temperature visualizer lets you drag all three at once.

One practical note for anyone working from an API response rather than raw scores: most providers return log-probabilities, not logits and not probabilities. A logprob is already post-softmax, so converting it back is a plain exp() rather than another softmax — running softmax over a set of logprobs is a common and quietly wrong step that produces a valid-looking distribution with the wrong shape. The logprob to probability converter handles that direction. And if your problem has independent labels rather than mutually exclusive ones — tagging an image with every object it contains, say — softmax is the wrong function entirely; you want a per-label sigmoid instead, because forcing the outputs to sum to 1 makes two true labels compete for the same budget.

Every example below was computed by hand first and then reconciled against the code that powers the calculator, so you can follow the arithmetic step by step and reproduce it on paper. Exponentials are shown to six decimal places and probabilities to four.

The three temperature examples share one input vector on purpose. Compare them and the pattern is clear: the winning class never changes, only how much of the probability mass it holds — 0.864 at T = 0.5, 0.659 at T = 1, 0.502 at T = 2.

1. Paste your logits. Type them into the first box separated by commas, spaces, tabs or new lines. Square brackets are stripped automatically, so a value copied straight out of a Python list or a NumPy array prints fine as-is. Negatives and decimals are accepted; the vector is capped at 100 values.
2. Set the temperature. Leave the slider at 1.00 for the plain softmax. Push it above 1 to flatten the distribution or below 1 to sharpen it. The five preset buttons load the same input vector at different temperatures, which is the quickest way to see the effect.
3. Name your classes (optional). Enter a comma-separated list such as cat, dog, bird to replace the auto-generated “Class 1…n” labels. The count has to match the number of logits; if it does not, the tool says so and falls back to auto-labels rather than silently misaligning them.
4. Read the results. The four tiles give the predicted class, the probability sum (a running proof the maths is right), the entropy in bits, and a sharpness percentage. Below them the bar chart and per-class table show every intermediate quantity — the scaled score z/T, the shifted exponential, the probability and the percentage.
5. Check the working, then export. The step-by-step panel prints the actual denominator used for your input, so you can verify the division by hand. The copy buttons emit a comma list, a JSON array or a Markdown table at 2, 4 or 6 decimal places.

If you are checking this page against your own implementation, these are the three lines that matter. The NumPy version is the stable formula written out in full; the PyTorch version is the one you should actually ship, because it is fused and differentiable.

import numpy as np

def softmax(z, T=1.0):
    s = np.asarray(z, dtype=float) / T     # 1. temperature scaling
    s = s - s.max(axis=-1, keepdims=True)  # 2. stability shift
    e = np.exp(s)                          # 3. exponentiate
    return e / e.sum(axis=-1, keepdims=True)   # 4. normalise

softmax([2.0, 1.0, 0.1])        # -> [0.6590, 0.2424, 0.0986]
softmax([2.0, 1.0, 0.1], T=2.0) # -> [0.5017, 0.3043, 0.1940]

# PyTorch equivalent (already numerically stable):
import torch
torch.softmax(torch.tensor([2.0, 1.0, 0.1]) / 1.0, dim=-1)

Two details are easy to get wrong. First, the axis argument: on a batch of shape (batch, classes) you almost always want axis=-1 with keepdims=True, otherwise the broadcast silently normalises across the wrong dimension and every row still sums to 1 — a bug that produces plausible-looking nonsense. Second, do not apply softmax before a loss function that expects raw logits. PyTorch's CrossEntropyLoss and TensorFlow's from_logits=True path both run the softmax internally; feeding them probabilities applies it twice and quietly flattens your gradients. This calculator is for inspecting and reporting a distribution, not for use inside a training loop.

Masking, padding, and the log-sum-exp trick

Real vectors usually have entries that must be excluded — padding positions in a batch, future tokens behind a causal mask, vocabulary entries banned by a constraint. The correct way to remove one is to set its logit to negative infinity before the softmax, not to zero its probability afterwards. At −∞ the stability shift makes the exponent −∞, so exp returns an exact 0, the class contributes nothing to the denominator, and the survivors renormalise on their own. Zeroing afterwards leaves the masked mass in the denominator and every remaining probability comes out too small.

The failure case is a row where everything is masked. Then max(s) is itself −∞, the shift computes −∞ − (−∞) = NaN, and one poisoned row spreads NaN through the rest of the batch on the next matrix multiply. Fully-padded rows are the usual culprit, and the standard fixes are to use a large finite sentinel such as −1e9 instead of true −∞, or to drop the empty rows before the softmax. If you only want to discourage a token rather than forbid it, a finite offset is the right instrument — the logit bias calculator shows what a given bias does to the resulting probabilities. Note that this page's own input box rejects inf and NaN tokens by name rather than accepting them, so the mask behaviour described here is something to check in your code, not something to reproduce in the box above.

Finally, when you need the denominator rather than the probabilities — for a loss, a log-likelihood or a perplexity — do not compute log(sum(exp(z))) directly. Use the log-sum-exp identity, log Σ exp(zⱼ) = m + log Σ exp(zⱼ − m) with m = max(z), which is the same stability shift applied one level up and is available as scipy.special.logsumexp and torch.logsumexp. It is the same reason log-softmax exists, and the reason a well-written training loop never materialises the probabilities at all.

• Goodfellow, Bengio & Courville — Deep Learning (MIT Press, 2016), §4.1 & §6.2.2
• PyTorch documentation — torch.nn.Softmax (reference implementation)
• Hinton, Vinyals & Dean — Distilling the Knowledge in a Neural Network (2015), §2 (temperature)

The formulas on this page were last cross-checked against these sources and PyTorch on 2026-08-17. Softmax is a stable mathematical definition, so this tool needs no rate or schedule updates — only the worked examples are periodically re-reconciled.