Local AI helper
VRAM calculator
Pick your machine and a model people actually run — then see weights + KV cache for your context, activations, and runtime overhead. Download pages often ignore the cache; here’s what KV cache is.
Typical desktop or workstation with a discrete GPU. GGUF models run in Ollama, LM Studio, or llama.cpp.
Top models · Chat · x86 PC
Curated from what people pull on Ollama / GGUF — not every library tag.
Tune
Weights locked from catalog (8B). Change context below — that’s usually what breaks “it fits” claims.
Longer context = larger KV cache (the coral bar). That’s the model’s working memory for this chat—not the model file on disk. See What is KV cache? below.
Estimated total
7.35 GB
Fits with room · 12.0 GB available · 4.65 GB free
Max context here: 38k
Llama 3.x 8B-class (scaled to 8B)
- Weights4.25 GB (58%)
- KV cache (context)1.00 GB (14%)
- Activations / scratch0.76 GB (10%)
- Runtime + driver1.35 GB (18%)
Right now in this estimate
Library “needs ~X GB” figures are mostly weights (the model file). At 8.2k context, your KV cache alone is 1.00 GB.
- Architecture used: Llama 3.x 8B-class (scaled to 8B) — 32 layers, hidden 4096, GQA 32→8 KV heads.
What is KV cache?
When a chat model answers, it doesn’t re-read your whole conversation from scratch on every new word. It keeps a Key–Value (KV) cache: a packed summary of attention for every token already in the window (your prompt, pasted code, prior replies). That cache lives in GPU / unified memory while the chat is running.
Weights vs KV (simple)
- Weights — the model itself (the big file you download). Size is mostly fixed for a given quant.
- KV cache — scratch pad for this conversation. Grows as the context window grows.
- Same 14B model at 4k context can look fine on 12 GB; at 64k the cache alone may not fit.
What makes KV grow
- More tokens — longer chats, big pastes, “128k context” settings.
- Deeper / wider models — more layers and attention heads → more cache per token.
- Batch — two chats at once ≈ roughly two caches.
- KV precision — FP16 cache uses more memory than quantized KV (when your runtime supports it).
Why download pages mislead: “Needs 10 GB VRAM” usually means weights (often Q4) with a short default context. They rarely include a long coding session, a whole PDF in the prompt, or LM Studio’s extra overhead. This calculator adds those pieces so you can see the coral KV bar before you hit out of memory.
Under the hood we estimate: KV ≈ 2 (Keys + Values) × layers × KV heads × head size × context tokens × batch × bytes per value — adjusted for GQA (grouped-query attention), which most modern models use to keep cache smaller than older designs.
Model lists
x86 PC — common Ollama/GGUF pulls for 8–24 GB cards (plus a few larger for big GPUs). Spark / Blackwell — gpt-oss, 70B+, Nemotron, large coders on unified memory. Mac — MLX community quants for LM Studio / mlx-lm. Tags must exist in our synced catalogs or they won’t show.