AI Compute-Optimal (Chinchilla) Calculator

For a dense transformer, training compute in FLOPs is well approximated by C ≈ 6·N·D — where N is the parameter count and D is the number of training tokens. The factor 6 comes from Kaplan et al. (2020): about 2 FLOPs per parameter per token on the forward pass and 4 on the backward pass. Attention and embedding costs are small at scale and folded into the approximation.

Hoffmann et al. (2022) — the “Chinchilla” paper — trained hundreds of models across compute budgets and found that loss is minimised, for a fixed budget, when the token count is roughly 20 times the parameter count. Substituting D = 20·N into the compute relation gives C = 120·N², so the calculator solves:

• N_opt = √(C / 120)
• D_opt = 20 × N_opt

Budgets given in PFLOP-days are converted with 1 PFLOP-day = 10¹⁵ × 86,400 = 8.64×10¹⁹ FLOPs. GPU-hours are converted from the cluster spec: C = peak_FLOP/s × GPUs × hours × 3600 × (MFU / 100), where MFU (Model FLOPs Utilization) discounts peak throughput to the fraction a real training loop sustains — typically 35–55%.

Check mode estimates the loss of a planned model with Chinchilla's parametric fit L(N,D) = 1.69 + 406.4/N^0.34 + 410.7/D^0.28. The first term is the irreducible entropy of natural text; the second penalises too few parameters; the third penalises too little data. It then classifies the token-to-parameter ratio: below ~15 is under-tokened, ~15–25 is compute-optimal, and above ~25 is over-tokened (common and often deliberate for inference-cheap models). Every result is cross-checked by recomputing 6·N·D and confirming it matches the input budget.

• Hoffmann et al. (2022) — Training Compute-Optimal Large Language Models (Chinchilla), arXiv:2203.15556
• Kaplan et al. (2020) — Scaling Laws for Neural Language Models, arXiv:2001.08361
• NVIDIA H100 datasheet — BF16 peak throughput (used for the GPU-hours default)

Constants (the 20:1 ratio, the 6ND factor, and the loss-fit parameters E, A, α, B, β) were last cross-checked against the Chinchilla and Kaplan papers on 2026-07-10. This tool covers dense transformer pre-training only.