LLM Training Time & Cost Calculator

Training compute is dominated by one quantity: the number of floating-point operations (FLOPs) needed to push every training token through the model, forward and backward. The widely-used approximation is C ≈ 6 × N × D, where N is the parameter count and Dis the number of training tokens. The factor 6 breaks down as 2 FLOPs for the forward pass and 4 for the backward pass, per parameter per token. This is the form established by Kaplan et al. (2020) and reused by DeepMind's Chinchilla paper (2022) as the standard way to budget a training run.

The calculator then turns that compute figure into time and money:

1. Total compute. C = 6 × N × D FLOPs, with N and D converted from billions to absolute counts.
2. Per-GPU throughput. achieved = peak × (MFU / 100). Peak comes from the GPU's dense BF16/FP16 datasheet rating (for example H100 SXM at 989 TFLOP/s). MFU — Model FLOPs Utilization — is the fraction of peak you actually sustain in practice.
3. Cluster throughput. aggregate = num_gpus × achieved, assuming near-linear scaling. Very large clusters lose a few percent to communication, so treat this as an upper bound.
4. Wall-clock time. t = C / aggregate seconds, then converted to hours and days.
5. GPU-hours. gpu_hours = num_gpus × t / 3600 — the unit cloud providers bill. Notably this equals C / (peak × MFU) / 3600, independent of the GPU count, so the tool cross-checks the two formulas against each other.
6. Cost. cost = gpu_hours × price_per_gpu_hour.
7. Petaflop/s-days. C / (1e15 × 86400) — the standard unit for comparing runs against published models. GPT-3 famously came to ≈ 3,640.

Three assumptions are worth stating plainly. First, 6ND ignores attention and activation FLOPs, which are a small correction at these scales. Second, MFU is a user input because it depends on model shape, parallelism strategy, and framework — there is no single right value. Third, GPU scaling is assumed near-linear. These are the same caveats used by public estimators from EpochAI and MosaicML. The tool does not model VRAM feasibility or parallelism topology — see the related training-memory and mixture-of-experts calculators for those.

• Kaplan et al. — Scaling Laws for Neural Language Models (2020) — the 6ND law
• Hoffmann et al. (Chinchilla) — Training Compute-Optimal LLMs (2022)
• NVIDIA H100 Tensor Core GPU datasheet — peak BF16/FP16 throughput
• NVIDIA A100 datasheet — BF16 dense throughput
• Chowdhery et al. — PaLM (2022) — empirical MFU figures
• Brown et al. — GPT-3 (2020) — the ~3,640 petaflop/s-days cross-check

GPU throughput figures and the compute formulas on this page were last cross-checked against the sources above on 2026-07-18. All GPU numbers are dense (no 2:1 sparsity). No live pricing is fetched — the cost-per-GPU-hour field is a user input so your own quote drives the estimate.