How much RAM do I need to run a 70B model on a Mac?

A 70B model at Q4_K_M needs roughly 40 GB just for weights, plus KV cache and overhead — about 43 GB in total at 8K context. Because macOS only lets the GPU use ~67–75% of unified memory, you realistically want a 64 GB Mac (48 GB of usable GPU memory), or a 48 GB Mac if you drop to Q3/Q2. A 32 GB Mac cannot run 70B at any usable quant.

Can a MacBook Air M2 with 16 GB run Llama 3.1 8B?

Yes, comfortably. Llama 3.1 8B at Q4_K_M needs about 6.5 GB (4.5 GB weights + 1 GB KV cache at 8K context + 1 GB overhead). A 16 GB Mac gives the GPU roughly 10.7 GB at the macOS default cap, so it fits with headroom to spare. Expect around 12–17 tokens/sec on the base M2.

Why can't my Mac use all its unified memory for the GPU?

Unified memory is shared between the CPU, GPU, and the operating system. macOS reserves a slice for the OS and other apps, so Metal reports a lower recommendedMaxWorkingSetSize than the total RAM. Ollama caps GPU allocation at ≈2/3 of unified memory at 36 GB or less, and ≈3/4 above that. You can raise the cap on macOS Sonoma or later with the iogpu.wired_limit_mb sysctl.

Is a Mac good for running local LLMs?

For inference, yes. Unified memory lets even a mid-range Mac hold models that would need a multi-thousand-dollar GPU, and Apple's memory bandwidth (100–546 GB/s depending on chip) gives usable decode speeds. Macs are weaker for training and for very high concurrent throughput, where NVIDIA GPUs still lead. For a single user running Ollama, LM Studio, or MLX, an Apple Silicon Mac is one of the easiest ways to run models locally.

What is the biggest LLM an M4 Max 128 GB can run?

At the macOS default cap you get about 96 GB of usable GPU memory. That comfortably runs a 70B model at Q6_K or Q8_0, and fits a 72B at high quality. You can load quantized versions of models up to roughly 120B parameters at Q4, though quality and speed both drop as the model grows. The biggest-model table in the calculator lists the exact largest class per quant for your settings.

Does quantization change how much memory a model needs?

Yes — quantization is the main lever. Weights scale linearly with bits-per-weight: Q4_K_M (~4.83 effective bpw) is about 30% the size of F16 (16 bpw), and Q2_K (~3.35 bpw) is smaller still. Halving the bits roughly halves the weight memory, at the cost of some quality. The KV cache and overhead do not shrink with weight quantization, so at long context they become a larger share.

How accurate is the tokens-per-second estimate?

It is a bandwidth-bound roofline, not a benchmark. Decoding one token requires reading every weight from memory once, so speed is approximately memory bandwidth divided by model size. Real throughput is lower because of attention overhead and imperfect bandwidth use, so the tool shows a band (roughly 50–80% of the roofline). Treat it as a ballpark, not a promise; measure with your own runtime for exact numbers.

Does this cover Intel Macs or training memory?

No. This tool is for Apple Silicon (M1–M4) inference only. Intel Macs and AMD eGPUs use discrete VRAM, not unified memory, so the cap model does not apply. For fine-tuning memory, use the LLM VRAM calculator, which adds gradients and optimizer state. This calculator answers one question: will the model load and run on your Mac.

When was this last verified?

The chip specs, memory-bandwidth figures, quant sizes, and the macOS GPU-cap heuristic were last cross-checked against Apple's specifications, the Ollama FAQ, and published GGUF file sizes on 2026-07-09.

AI · Local models

Mac / Apple Silicon LLM Calculator — Can I Run This Model?

Tells you whether an open LLM will actually run on your Apple Silicon Mac. It models the macOS unified-memory GPU cap — the part that trips people up — along with the model's quantized weight size, KV cache, and runtime overhead, then gives a clear verdict and the biggest model your Mac can handle.

By Induwara Ashinsana— Executive Director, Ryzera TechnologiesUpdated Jul 9, 2026

Can I run this model?

Unified-memory aware

Mac chip

Unified memory

Model

Quantization

Context length (tokens)

GPU memory cap67% · 10.7 GB usable

Runs comfortablyLlama 3.1 8B · Q4_K_M · 8,192 ctx

Weights 4.5 GBKV cache 1.0 GBOverhead 1.0 GBUsable cap 10.7 GB

Required

6.5 GB

Usable GPU memory

10.7 GB

Free after load

4.2 GB

for other apps + longer context

Est. speed

10–17 tok/s

bandwidth-bound estimate (M2, 100 GB/s)

Memory component	Size
Model weights (Q4_K_M, 4.83 bpw)	4.5 GB
KV cache (fp16, 8,192 ctx)	1.0 GB
Runtime overhead	1.0 GB
Total required	6.5 GB

Best quant that fits this Mac at these settings: Q8_0 — near-lossless; indistinguishable from f16.

Biggest model this Mac can run

Quant	Effective bpw	Largest that fits
F16	16	3B
Q8_0	8.5	8B
Q6_K★	6.56	8B
Q5_K_M★	5.67	8B
Q4_K_M★	4.83	8B
Q4_0	4.5	14B
Q3_K_M	3.9	14B
Q2_K	3.35	14B

Sources: Apple Mac specs (chip memory + bandwidth), Apple Metal recommendedMaxWorkingSetSize, Ollama macOS GPU-memory FAQ, and llama.cpp GGUF quant sizes. Speed is a bandwidth-bound estimate, not a benchmark. Full citations are listed below the calculator.

How it works

On an Apple Silicon Mac there is no separate graphics card. The CPU, GPU, and Neural Engine share one pool of unified memory. That is what lets a Mac hold large models cheaply — but it also means the GPU cannot use all of the RAM, because macOS keeps a slice for the operating system and everything else you have open. Apple exposes this budget to apps as Metal's recommendedMaxWorkingSetSize. Runtimes like Ollama translate it to a rule of thumb: give the GPU about two-thirds (≈67%) of unified memory on machines up to 36 GB, and about three-quarters (≈75%) above that.

The calculator estimates four numbers and compares them:

Weights. Model size in bytes is params × bpw ÷ 8, where bpw is the effective bits-per-weight of the GGUF quant. We use measured real-file values (Q4_K_M ≈ 4.83, Q8_0 ≈ 8.5, Q2_K ≈ 3.35), which run a little above the nominal block sizes because embedding and output tensors stay at higher precision — so estimates match the file sizes Ollama and Hugging Face publish.
KV cache. Attention stores a key and value for every token at every layer: 2 × layers × kv_heads × head_dim × context × 2 bytes. Modern models use grouped-query attention, which keeps far fewer KV heads and makes this term small until the context gets long.
Overhead. A fixed ≈1 GB for the compute graph and runtime buffers.
Usable GPU memory. unified_memory × cap%. The verdict compares required (weights + KV + overhead) against this: at or below 90% of usable it “Runs comfortably”, up to 100% it “Runs, but tight”, and above it “Won't fit”.

The speed figure is a memory-bandwidth roofline: decoding a token reads every weight once, so tokens per second is roughly bandwidth ÷ model_size. Real throughput lands lower, so the tool shows a band and labels it an estimate. You can raise the GPU cap yourself on macOS Sonoma or later with sudo sysctl iogpu.wired_limit_mb, which the tool pre-fills when a model is tight or over budget.

Worked examples

Runs comfortably

MacBook Air M2 · 16 GB · Llama 3.1 8B · Q4_K_M · 8K context

Weights: 8.03B × 4.83 ÷ 8 = 4.85 GB ÷ 2³⁰ = 4.52 GB
KV cache: 2 × 32 × 8 × 128 × 8192 × 2 B = 1.00 GB
Overhead: 1.00 GB → Required = 6.52 GB
Usable: 16 × 67% = 10.72 GB (macOS default cap)
6.52 ≤ 0.9 × 10.72 (9.65) → Runs comfortably, ~4.2 GB free

Won't fit

Mac Studio M2 Max · 32 GB · Llama 3.3 70B · Q4_K_M · 8K context

Weights: 70.6B × 4.83 ÷ 8 = 42.6 GB ÷ 2³⁰ = 39.7 GB
KV cache: 2 × 80 × 8 × 128 × 8192 × 2 B = 2.5 GB
Overhead: 1.0 GB → Required = 43.2 GB
Usable: 32 × 67% = 21.44 GB
43.2 > 21.44 → Won't fit; even Q2_K (27.5 GB weights) exceeds it
Fix: choose a 64 GB Mac, or drop to a 32B model at Q4

Runs, but tight

Mac mini M4 · 32 GB · Qwen2.5 32B · Q4_K_M · 4K context

Weights: 32.8B × 4.83 ÷ 8 = 19.8 GB ÷ 2³⁰ = 18.44 GB
KV cache: 2 × 64 × 8 × 128 × 4096 × 2 B = 1.00 GB
Overhead: 1.00 GB → Required = 20.44 GB
Usable: 32 × 67% = 21.44 GB
0.9 × 21.44 (19.30) < 20.44 ≤ 21.44 → Runs, but tight (~1 GB free)

Frequently asked questions

Sources & references

Chip specs, bandwidth figures, quant sizes, and the macOS GPU-cap heuristic were last cross-checked on 2026-07-09. Weight estimates reconcile to published GGUF download sizes within about 8%. Speed is a bandwidth-bound estimate, not a benchmark.

Related tools

LiveAI

LLM VRAM Calculator

Estimate the GPU VRAM needed to run or fine-tune any open LLM (Llama 3, Mistral, Qwen, Gemma, DeepSeek) at a given precision, context, and batch size — and check whether it fits your GPU. Formulas cited, runs in your browser.

Open tool

LiveAI

Small LLM Comparison

Compare small, self-hostable open-weight LLMs — Llama 3.2, Gemma 3, Qwen3, Phi-4-mini, SmolLM2, Ministral, IBM Granite and Falcon 3 — by parameters, context window, license and modality. Enter your RAM or GPU VRAM and desired context to see which models actually fit your machine, with an estimated minimum memory and a source-cited recommendation. Free, no signup, every spec cited to the official model card.

Open tool

LiveAI

AI Model Size Calculator

Calculate the exact on-disk file size of any LLM at FP32, FP16, 8-bit, 4-bit, or GGUF quants like Q4_K_M — and how long it downloads on your connection. Compare every quant side by side. Free, no signup, sources cited.

Open tool

Rate this tool

Be the first to rate

Comments & feedback

Spotted a bug or want an improvement? Tell us — our team reviews every comment, and good ideas get built. Comments are public and anonymous.

Found a bug, edge case, or want another chip or model added?

Email me at [email protected] — most fixes ship within 24 hours.