LLM GPU Memory (VRAM) Calculator

Every figure is computed in bytesfrom published transformer-memory formulas, then shown in GiB (1 GiB = 2³⁰ bytes). The calculator never downloads or runs a model — it reads each model's parameter count and attention shape from its official model card and applies the closed-form equations below.

1. Model weights

Weight memory is the parameter count P times bytes-per-weight: FP32 = 4, FP16/BF16 = 2, INT8 = 1, and INT4/Q4 = 0.5. So an 8B model is ~16 GiB at FP16 but only ~4 GiB at 4-bit. (EleutherAI, Transformer Math 101.)

2. KV cache (inference)

During generation the model caches a key and value for every token and every layer: 2 × batch × tokens × layers × (KV heads × head dim) × 2 bytes. Grouped-query attention (GQA) — used by Llama 3, Mistral, and Qwen — uses far fewer KV heads than query heads, which is why their cache stays small even at long context. DeepSeek 7B keeps full multi-head attention, so its cache is larger for the same size.

3. Training memory

Each trainable parameter under mixed-precision AdamW costs 16 bytes: a 2-byte FP16 weight, a 2-byte FP16 gradient, a 4-byte FP32 master copy, and 8 bytes of Adam moment and variance. 8-bit Adam cuts the optimizer states to give 10 bytes/param; plain SGD drops the moments for 8 bytes/param. Full fine-tuning applies this to every parameter (≈120 GiB for 8B). LoRA freezes the base in your chosen precision and trains a small adapter (~0.5% of parameters) at those rates. QLoRA freezes the base at 4-bit and trains the adapter on top, which is how an 8B fine-tune drops to single-digit GiB. (EleutherAI; Dettmers et al., QLoRA.)

4. Activations & overhead

For training, activation memory with gradient checkpointing is estimated as 2 × batch × tokens × hidden size × layers, plus a fixed ~1 GiB CUDA-context floor. For inference the tool follows Hugging Face's rule of thumb and adds the larger of 1 GiB or 20% of the weight size. Because real frameworks vary, the headline shows a ±15% planning range — size for the top of it.

5. Fit check & cross-check

The total is compared against the chosen GPU's VRAM; if it overflows, the tool reports how many cards you need. As an independent check, the same inference estimate is run against Hugging Face's ≈ 1.2× model sizeheuristic and the two are shown side by side — the same idea as cross-checking a tax figure against the regulator's own formula.

• EleutherAI — Transformer Math 101 (inference, KV cache, and training memory formulas)
• Dettmers et al. — QLoRA: Efficient Finetuning of Quantized LLMs
• Hugging Face — Accelerate Model Memory Estimator (cross-check multipliers)
• Meta — Llama 3 8B model card (parameters, layers, GQA config)

Model parameter counts, attention shapes, and GPU VRAM capacities were last verified against the sources above on 2026-06-05. Estimates are guidance, not a guarantee — plan for the upper end of the ±15% range.