Huber Loss Calculator (Smooth L1 Loss)

Huber loss was introduced by Peter J. Huber in “Robust Estimation of a Location Parameter” (1964) to combine the best of two common regression losses. Squared error (MSE) is smooth and easy to optimise but squares every mistake, so a single outlier can dominate training. Absolute error (MAE) resists outliers but has a kink at zero that makes its gradient jump. Huber loss is quadratic near zero and linear far from it, so it is both smooth and outlier-robust.

Work per element. For a predicted value ŷ and actual value y, the residual is a = ŷ − y. With threshold δ > 0, Huber loss is the piecewise function (Huber 1964; torch.nn.HuberLoss):

L(a) = 0.5·a²  if |a| ≤ δ   ·   δ·(|a| − 0.5·δ)  if |a| > δ

The two pieces meet at |a| = δ with matching value and slope, so the curve is continuous and differentiable there — that C¹ smoothness is what separates Huber from a raw clipped MAE. PyTorch's Smooth L1 loss is the same shape rescaled by its threshold β:

SmoothL1(a) = 0.5·a²/β  if |a| < β   ·   |a| − 0.5·β  if |a| ≥ β

1. Region split. Each residual is tagged quadratic (|a| ≤ δ) or linear (|a| > δ). Sweeping δ moves points between the two, which is how you trade smoothness against outlier-robustness. The tool shows the count and percentage in each region.
2. Reduction. Following PyTorch, mean returns (1/N)·Σ Lᵢ (the training objective), sum returns Σ Lᵢ, and none keeps the per-element vector.
3. Comparison metrics. The same residuals also give MSE = (1/N)·Σ a² and MAE = (1/N)·Σ |a|, printed alongside so you can see Huber sitting between them.
4. The Huber–MSE identity. When every residual is quadratic, the mean Huber loss equals exactly 0.5·MSE. The tool flags this and shows the half-MSE value so you can confirm it.

The documented relationship between the two losses is SmoothL1(β) = HuberLoss(δ = β) / β, so they coincide only when β = δ = 1. To keep results trustworthy the tool recomputes every Smooth L1 element through this identity and confirms it matches the direct formula to floating-point precision before displaying anything. All arithmetic runs client-side in double precision.

• Huber, P. J. (1964) — Robust Estimation of a Location Parameter, Ann. Math. Statist. 35(1): 73–101 (original piecewise definition)
• PyTorch — torch.nn.HuberLoss (δ-parameterised formula and reduction semantics)
• PyTorch — torch.nn.SmoothL1Loss (β form and the SmoothL1(β) = Huber(δ)/β identity)
• TensorFlow / Keras — tf.keras.losses.Huber (same formula, default δ = 1.0 cross-check)

The formulas on this page were last cross-checked against these sources on 2026-07-12. Huber and Smooth L1 loss are fixed mathematical definitions, so this tool needs no rate or schedule updates — only the worked examples are periodically re-reconciled.