AI Model Size & Quantization Calculator

A model file is just its weights written to disk, so its size follows one identity from Hugging Face's model-memory anatomy: the number of parameters multiplied by the number of bytes used to store each one.

sizeBytes = parameters × (bitsPerWeight ÷ 8)

The only variable that changes between formats is bits per weight. Full precision (FP32) uses 32 bits, 4 bytes, per parameter. The released half-precision weights (FP16 or BF16) use 16 bits, or 2 bytes. Quantization trades a little quality for a much smaller file by storing each weight in fewer bits:

• FP32 → 32 bits (4 bytes/param)
• FP16 / BF16 → 16 bits (2 bytes/param)
• FP8 / INT8 → 8 bits (1 byte/param)
• INT4 → 4 bits (0.5 byte/param)
• GGUF k-quants → Q8_0 ≈ 8.5, Q6_K ≈ 6.56, Q5_K_M ≈ 5.67, Q4_K_M ≈ 4.83, Q3_K_M ≈ 3.91, Q2_K ≈ 2.96 bits

The floating-point and integer figures come straight from the PyTorch torch.dtype element-size table. The GGUF k-quant numbers are the documented effective bits per weight from llama.cpp. They are not round numbers because each k-quant keeps small per-block scale and minimum values, and leaves a few sensitive tensors at higher precision, so the true average lands a bit above the nominal bit width.

Two presentation details matter. First, sizes appear in both GB (decimal, 10⁹ bytes, how vendors quote sizes) and GiB (binary, 2³⁰ bytes, what your operating system's disk meter shows); a 140 GB model reads as roughly 130 GiB in your file manager. Second, an optional ~3% overhead can be added for the tokenizer, config, and safetensors/GGUF metadata shipped alongside the weights. It is toggleable so the core weights-only math stays exact and easy to verify.

Download time uses the standard link-rate identity: seconds = file bits ÷ link bits-per-second = (sizeBytes × 8) ÷ (speedMbps × 10⁶). The default speed is 25 Mbps, Sri Lanka's median fixed-broadband speed from the Ookla Speedtest Global Index, because on a metered or modest connection the total gigabytes and the wait both decide whether a quant is worth pulling. Remember this is disk and download size only, running a model needs more GPU memory than the file size, which the LLM VRAM Calculator handles.

Edge cases the formula handles, and the ones it can't

• Rounded model names. “8B” is a label, not an exact count, Llama 3 8B is about 8.03 billion parameters. Use the exact figure from the model card when you need the last decimal; at Q4_K_M, 8.03B gives 4.85 GB instead of 4.83 GB.
• Mixture-of-Experts models. Enter the total parameter count. Mixtral 8x7B stores all 46.7B parameters on disk even though only 12.9B are active per token, so the file follows the total.
• Sharded downloads. Large safetensors and GGUF releases are split into several files (for example model-00001-of-00030.safetensors). The result here is the sum of all shards.
• Training checkpoints. A checkpoint that also stores AdamW optimizer states carries about 8 extra bytes per parameter (Hugging Face's memory anatomy guide), so it can be several times the size of the inference weights this tool measures.
• Real GGUF files drift slightly. llama.cpp chooses tensor types per layer, and embedding and output layers differ between architectures, so a published file can land a few percent above or below the average bits-per-weight estimate. Treat the number as a planning figure to within a few percent, not a byte-exact checksum.
• Zero or blank input. A parameter count of 0 returns 0 bytes and the download time shows “—” rather than an error.

The file size decides two practical things: whether the model fits on your disk and in your memory, and how much of your data allowance the download uses. A useful way to choose is to start from the hardware you have and work backwards.

The weights have to fit in RAM (for CPU inference, or unified memory on a Mac) or in VRAM (for a GPU), with room left over for the context cache and your operating system. As a starting point, using the file sizes this calculator produces:

• 8 GB laptop RAM: a 7B model at Q4_K_M is 7e9 × 4.83 ÷ 8 = 4.23 GB, which leaves a few gigabytes for everything else. Larger quants get tight.
• 16 GB RAM: a 14B model at Q4_K_M (8.45 GB) or an 8B model at Q8_0 (8.50 GB) is a reasonable ceiling.
• 24 GB GPU: a 27B model at Q4_K_M is 16.30 GB and a 32B model is 19.32 GB, both fit as files, but long contexts eat the remaining headroom.

File size is only the floor. The LLM VRAM Calculator adds the KV cache for your context length, and Mac users can check unified-memory limits with the Apple Silicon LLM Calculator.

For quality, Q4_K_M is the usual default for local use: about 30% of the FP16 size with a small quality loss. Step up to Q5_K_M, Q6_K or Q8_0 when you have spare memory and care about accuracy on code or maths; drop to Q3_K_M or Q2_K only when nothing else fits. GGUF is not the only format, GPTQ, AWQ and EXL2 target GPUs differently, and the quantization method comparison explains when each is the better pick. Which files you need also depends on the runtime; the local LLM runtime comparison covers Ollama, LM Studio, llama.cpp and vLLM.

On a capped connection, check the size here before you start, download during an off-peak or unlimited window if your plan has one, and use a downloader that can resume, huggingface-cli download and ollama pull both pick up an interrupted download instead of starting again.

• PyTorch: torch.dtype element sizes (float32/16, bfloat16, float8, int8)
• Hugging Face: Model Memory Anatomy (size = parameters × bytes per parameter)
• llama.cpp: quantize README (GGUF k-quant types and effective bits per weight)
• llama.cpp PR #1684: k-quants introduction and bits-per-weight notes
• Ookla Speedtest Global Index: Sri Lanka median fixed-broadband speed

The bits-per-weight constants used here were last cross-checked against the PyTorch dtype table and the llama.cpp quantize README on 2026-06-12. Sizes are computed in bytes and shown in both decimal GB and binary GiB.