On a 32 GB GPU, 12 catalog models run fully on the GPU at an 8,192-token context. The most capable is Qwen3.6-35B-A3B at Q4_K_M (needs ~23.9 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 32 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
32 GB buys headroom on the 30B class and a reach into the next one up. Dense 30B-class models fit at 4-bit sweet-spot quants with genuine slack, models just below that class step up to near-lossless quants, and the larger mixture-of-experts models in the upper-40B range come into reach at compact 4-bit quants. The list below is derived live from the fit engine, largest first.
This tier's real decision isn't which model fits — it's what to do with the surplus. Stretching to the largest entry uses most of the card and leaves little room for long context; running a 30B-class model with slack means higher quants, longer chats, and no out-of-memory anxiety. Coming from 24 GB, the honest framing is that 32 GB rarely changes which class you run — it changes how comfortably you run it.
Qwen3.6-35B-A3B at Q4_K_M · 36B params · needs ~23.9 GiB
One command to run it (llama.cpp):
llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| Qwen3.6-35B-A3B Qwen | Q4_K_M Runs fully on GPU | ~23.9 GiB |
| Qwen3-Omni-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen2.5-VL-32B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~22.9 GiB |
| DeepSeek-R1-Distill-Qwen-32B deepseek-ai | Q4_K_M Runs fully on GPU | ~22.9 GiB |
| Qwen3-32B Qwen | Q4_K_M Runs fully on GPU | ~22.8 GiB |
| gemma-4-31B-it google | Q4_K_M Runs fully on GPU | ~21.3 GiB |
| GLM-4.7-Flash zai-org | Q6_K Runs fully on GPU | ~26.3 GiB |
| Qwen3-Coder-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen3-30B-A3B-Instruct-2507 Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen3.6-27B Qwen | Q8_0 Runs fully on GPU | ~31.9 GiB |
| gemma-4-26B-A4B-it google | Q8_0 Runs fully on GPU | ~28.6 GiB |
| Devstral-Small-2-24B-Instruct-2512 mistralai | Q8_0 Runs fully on GPU | ~27.5 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Mostly comfort, and a little reach. The 30B class that just fits at 24 GB fits here with real slack — enough for longer context or a step up in quant — and the larger mixture-of-experts models in the upper-40B range come into range at compact 4-bit quants. The class boundary moves less than the headroom does.
Not fully on the GPU at healthy quants — that class becomes realistic at the 48 GB tier, and crushing it to very low bits usually gives worse output than a well-quantized 30B-class model in the same footprint. Partial offload to system RAM works for patient, occasional use, but it's slow; this page lists only full-GPU fits.
As a rule of thumb, parameter count wins — a larger model at a healthy 4-bit quant usually beats a smaller one at 8-bit. But 32 GB is where the exception gets interesting: a model you already trust at a near-lossless quant with generous context headroom is a very dependable daily driver. Test both on your actual task.
Yes — the KV cache grows with context, and it grows fastest on the biggest models, which is exactly where this tier's tightest fits live. The verdicts here hold at the stated context; for long-document or agent-style work, verify your model and target context in the fit calculator first.
None in particular — fit depends on memory size, so these verdicts apply to any 32 GB card. Speed doesn't transfer the same way: memory bandwidth varies between cards with the same VRAM, so check your specific card's GPU page for what generation speed to expect.
selected to compare · pick at least 2