AI Compute-Optimal (Chinchilla) Calculator

For a dense transformer, training compute in FLOPs is well approximated by C ≈ 6·N·D , where N is the parameter count and D is the number of training tokens. The factor 6 comes from Kaplan et al. (2020): about 2 FLOPs per parameter per token on the forward pass and 4 on the backward pass. Attention and embedding costs are small at scale and folded into the approximation.

Hoffmann et al. (2022), the “Chinchilla” paper, trained hundreds of models across compute budgets and found that loss is minimised, for a fixed budget, when the token count is roughly 20 times the parameter count. Substituting D = 20·N into the compute relation gives C = 120·N², so the calculator solves:

• N_opt = √(C / 120)
• D_opt = 20 × N_opt

Budgets given in PFLOP-days are converted with 1 PFLOP-day = 10¹⁵ × 86,400 = 8.64×10¹⁹ FLOPs. GPU-hours are converted from the cluster spec: C = peak_FLOP/s × GPUs × hours × 3600 × (MFU / 100), where MFU (Model FLOPs Utilization) discounts peak throughput to the fraction a real training loop sustains, typically 35–55%.

Check mode estimates the loss of a planned model with Chinchilla's parametric fit L(N,D) = 1.69 + 406.4/N^0.34 + 410.7/D^0.28. The first term is the irreducible entropy of natural text; the second penalises too few parameters; the third penalises too little data. It then classifies the token-to-parameter ratio: below ~15 is under-tokened, ~15–25 is compute-optimal, and above ~25 is over-tokened (common and often deliberate for inference-cheap models). Every result is cross-checked by recomputing 6·N·D and confirming it matches the input budget.

• Hoffmann et al. (2022): Training Compute-Optimal Large Language Models (Chinchilla), arXiv:2203.15556
• Kaplan et al. (2020): Scaling Laws for Neural Language Models, arXiv:2001.08361
• NVIDIA H100 datasheet: BF16 peak throughput (used for the GPU-hours default)

Constants (the 20:1 ratio, the 6ND factor, and the loss-fit parameters E, A, α, B, β) were last cross-checked against the Chinchilla and Kaplan papers on 2026-07-10. This tool covers dense transformer pre-training only.