Can I Run This Model? — LLM VRAM Calculator
Find out whether your GPU can run a given large language model — locally, in your browser. Pick a model (Llama, Mistral, Qwen, Gemma, Phi, Mixtral, DeepSeek and more, or enter a custom parameter count) and your hardware (NVIDIA, Apple Silicon, or a custom VRAM + bandwidth), and it calculates the VRAM needed at every quantization level (FP16, Q8, Q6_K, Q5_K_M, Q4_K_M down to IQ2), whether it fits with your context length, the maximum context that fits, and a rough tokens-per-second estimate — then recommends the best quant. Real VRAM, KV-cache and memory-bandwidth math; nothing is uploaded.
How to Use
- Pick the <strong>model</strong> you want to run — Llama, Mistral, Qwen, Gemma, Phi, Mixtral, DeepSeek and others are built in, or choose <em>Custom</em> and type a parameter count in billions.
- Pick your <strong>hardware</strong> — an NVIDIA GeForce or data-center GPU, an Apple Silicon config, or <em>Custom</em> to enter your own VRAM and memory bandwidth.
- Set the <strong>context length</strong> you need with the slider (longer context uses more VRAM for the KV cache) and, optionally, quantize the KV cache to 8- or 4-bit to save memory.
- Read the <strong>verdict</strong>: whether it fits, at which quantization, how much VRAM it uses, the largest context that fits, and a rough <strong>tokens-per-second</strong> estimate.
- Use the <strong>quantization table</strong> to see every level at once — FP16 down to IQ2 — with the size, whether it fits, the estimated speed and a quality note, so you can pick the best trade-off for your card.
What it works out
The quantization ladder
Quantization is how a big model fits on a small card: each weight is stored in fewer bits, trading a little accuracy for a lot less memory. These are the common GGUF levels, from full precision down to the aggressive formats that squeeze a 70B model onto a 24 GB card.
About the Can I Run This Model? — LLM VRAM Calculator
Meet the Can I Run This Model? — LLM VRAM Calculator: a free, no-fuss tool for web development and data tasks with nothing to install and no sign-up. Find out whether your GPU can run a given large language model — locally, in your browser. Pick a model (Llama, Mistral, Qwen, Gemma, Phi, Mixtral, DeepSeek and more, or enter a custom parameter count) and your hardware (NVIDIA, Apple Silicon, or a custom VRAM + bandwidth), and it calculates the VRAM needed at every quantization level (FP16, Q8, Q6_K, Q5_K_M, Q4_K_M down to IQ2), whether it fits with your context length, the maximum context that fits, and a rough tokens-per-second estimate — then recommends the best quant. Real VRAM, KV-cache and memory-bandwidth math; nothing is uploaded.
How it works
Put each value in its box and read the answer as you go. Because it recalculates live, you can play with the inputs to see how each one moves the result — handy for checking your own working or planning ahead. Everything happens on your device, so it is fast and private.
Want the deeper story? The Knowledge Base explains the ideas behind the tools in more detail.
Frequently Asked Questions
How does it calculate the VRAM?
Three parts, the same way llama.cpp / vLLM do it. <strong>Model weights</strong> = parameters × bits-per-weight ÷ 8 (a 7B model at Q4_K_M ≈ 4 GB). <strong>KV cache</strong> = 2 × layers × KV-heads × head-dim × context × bytes — this is what grows with your context length, and models with Grouped-Query Attention (most modern ones) use far less of it. <strong>Overhead</strong> for the CUDA/compute context. It adds those up and compares against your usable VRAM.
What do the quantization levels mean?
Quantization shrinks each weight from 16 bits down to ~2–8 bits, trading a little quality for a lot less memory. <strong>Q4_K_M</strong> (~4.8 bits) is the community's sweet spot — most of the quality at a bit over a quarter of the size. Q6_K and Q8_0 are near-lossless if you have the room; Q3/Q2 and the IQ2 formats get you a huge model onto a small card but with real quality loss. The table shows the effective bits-per-weight and a quality note for each.
How accurate is the tokens/sec estimate?
It's a <strong>ballpark, not a benchmark.</strong> Text generation is mostly limited by memory bandwidth, so the estimate is roughly your GPU's bandwidth ÷ the size of the weights read per token, times a real-world efficiency factor. Actual speed depends on your backend (llama.cpp vs vLLM vs Transformers), flash-attention, batch size and driver overhead — treat it as an order-of-magnitude guide, especially for prompt processing which is compute-bound instead.
Does it handle Mixture-of-Experts models like Mixtral?
Yes. For an MoE model, <em>all</em> the parameters have to fit in memory (so Mixtral 8x7B needs room for ~47B weights), but only a couple of experts are active per token, so it <em>runs</em> at roughly the speed of a ~13B model. The calculator sizes the memory on the total and estimates speed on the active parameters.
What about Apple Silicon or running on CPU?
Apple Silicon uses <strong>unified memory</strong> shared between CPU and GPU — pick your chip and the tool uses roughly 72% of the RAM as GPU-usable (you can raise that limit in macOS). CPU + system-RAM options are included too, but expect single-digit to low-tens of tokens per second — the RAM bandwidth is a fraction of a GPU's.
Is my choice of model or hardware sent anywhere?
No. All the model specs, GPU specs and the math are built into the page and run entirely in your browser. Nothing you select is uploaded, and it works offline.
How do I use the Can I Run This Model? — LLM VRAM Calculator?
Just type your numbers. The answer shows up right away — there is no button to press. Change anything and it updates by itself.
Do I need to install or sign up for anything?
Not at all — it runs in the browser with nothing to install and no account. After it loads once, it even works without an internet connection.
Is my information private?
Yes. Everything happens in your browser. Nothing you type is sent to a server or saved anywhere.
Common Use Cases
Before you download
Check whether a model will actually run on your card before pulling a 40 GB file.
Pick the right quant
See every quantization at once and choose the best quality that fits your VRAM.
Plan a GPU purchase
Compare cards to see what you'd need for the model and context you want.
Size the context
Find the largest context window your hardware can hold for a given model.
Estimate speed
Get a rough tokens-per-second figure for a model + GPU combination.
Spec a server
Work out how much VRAM a deployment needs at your target precision and context.
Last updated: