On a 6 GB GPU, 9 catalog models run fully on the GPU at an 8,192-token context. The most capable is gemma-4-E4B-it at QAT-Q4_0 (needs ~5.9 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 6 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
6 GB of VRAM is the tight-budget tier, and it still runs real models. Today's small instruct models fit fully on the GPU at 4-bit sweet-spot quants, and the tiniest models run at near-lossless quants with room to spare. The list below is derived live from the fit engine, largest first — and at this tier, the entries lower down are often the smarter daily pick.
Take the largest small model that fits at 4-bit, or take a tiny model at a near-lossless quant and keep real headroom — at 6 GB that second option is more attractive than at any bigger tier, because the top entries use nearly the whole card. Margins are thin enough here that the KV cache, which grows with context, flips fits faster than anywhere else; and if a model you want doesn't appear, the honest fix is usually its smaller sibling rather than a crushed quant.
gemma-4-E4B-it at QAT-Q4_0 · 8B params · needs ~5.9 GiB
One command to run it (llama.cpp):
llama-server -m gemma-4-E4B_q4_0-it.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| gemma-4-E4B-it google | QAT-Q4_0 Runs fully on GPU | ~5.9 GiB |
| DeepSeek-R1-Distill-Qwen-7B deepseek-ai | Q4_K_M Runs fully on GPU | ~5.7 GiB |
| gemma-4-E2B-it google | Q8_0 Runs fully on GPU | ~5.8 GiB |
| Qwen3-Embedding-4B Qwen | Q4_K_M Runs fully on GPU | ~4.2 GiB |
| Phi-4-mini-instruct microsoft | Q8_0 Runs fully on GPU | ~5.7 GiB |
| SmolLM3-3B HuggingFaceTB | Q8_0 Runs fully on GPU | ~4.4 GiB |
| DeepSeek-R1-Distill-Qwen-1.5B deepseek-ai | Q8_0 Runs fully on GPU | ~2.7 GiB |
| Qwen3-Reranker-0.6B Qwen | BF16 Runs fully on GPU | ~2.6 GiB |
| Qwen3-Embedding-0.6B Qwen | Q8_0 Runs fully on GPU | ~2 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Yes, with realistic expectations. Small instruct models run fully on a 6 GB GPU at 4-bit sweet-spot quants and handle chat, drafting, and quick questions well. It's the tightest common tier, so headroom is scarce — but the small models that fit today are markedly better than what this class could do a year or two ago.
You can — llama.cpp and most desktop runtimes will split a model between GPU and CPU — but generation slows sharply once part of the model leaves the GPU, and at 6 GB most of a mid-size model ends up off-card. The table on this page lists only full-GPU fits; treat offloading as an occasional fallback, not a daily driver.
Usually because the card wasn't empty. On a 6 GB system the same GPU typically drives your display, and the OS, browser, and anything else GPU-accelerated hold VRAM before the model loads. Close heavy apps and try again, or step down one quant — the tightest entries here assume the model gets nearly the whole card.
The catalog also includes small utility models — embedders and rerankers for retrieval, and speech models — and because they're small, they show up prominently at this tier. The list ranks everything that fits by size; check a model's own page to see what kind of model it is before downloading.
Yes, and faster than at any larger tier — the KV cache grows with context, and 6 GB leaves the least slack to absorb it. A model that fits at this page's stated context can stop fitting well short of a long conversation, so check your model and target context in the fit calculator first.
selected to compare · pick at least 2