On a 24 GB GPU, 12 catalog models run fully on the GPU at an 8,192-token context. The most capable is Qwen3.6-35B-A3B at Q4_K_M (needs ~23.9 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 24 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
24 GB is the enthusiast tier where the 30B class runs fully on the GPU. Dense 30B-class models fit at the 4-bit sweet spot, mixture-of-experts models in that range fit with room to spare, and the mid-size models most people start with fit at high quants with generous context headroom. The list below is derived live from the fit engine, largest first.
This is the first tier where the choice between dense and mixture-of-experts (MoE) models really matters. Both classes fit, but they spend the memory differently: a dense model activates all of its parameters every token, while an MoE model of similar total size activates only a small fraction — so MoE entries in the list tend to feel noticeably quicker for their size. If responsiveness matters more than squeezing maximum quality per gigabyte, the MoE entries are worth a look first.
Qwen3.6-35B-A3B at Q4_K_M · 36B params · needs ~23.9 GiB
One command to run it (llama.cpp):
llama-server -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| Qwen3.6-35B-A3B Qwen | Q4_K_M Runs fully on GPU | ~23.9 GiB |
| Qwen3-Omni-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen2.5-VL-32B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~22.9 GiB |
| DeepSeek-R1-Distill-Qwen-32B deepseek-ai | Q4_K_M Runs fully on GPU | ~22.9 GiB |
| Qwen3-32B Qwen | Q4_K_M Runs fully on GPU | ~22.8 GiB |
| gemma-4-31B-it google | Q4_K_M Runs fully on GPU | ~21.3 GiB |
| GLM-4.7-Flash zai-org | Q5_K_M Runs fully on GPU | ~22.9 GiB |
| Qwen3-Coder-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen3-30B-A3B-Instruct-2507 Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen3.6-27B Qwen | Q4_K_M Runs fully on GPU | ~19.8 GiB |
| gemma-4-26B-A4B-it google | Q4_K_M Runs fully on GPU | ~18.4 GiB |
| Devstral-Small-2-24B-Instruct-2512 mistralai | Q4_K_M Runs fully on GPU | ~16.5 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
The 30B class, fully on the GPU at 4-bit sweet-spot quants — both dense models and the fast mixture-of-experts variants. It also means mid-size models run at high, near-lossless quants with real context headroom, which is why 24 GB cards are the default recommendation in most local-LLM communities.
MoE models activate only a fraction of their parameters per token, so they generally generate faster than a dense model of similar total size; dense models tend to squeeze more quality out of the same memory. If interactive speed matters, start with an MoE entry; if you want maximum capability per gigabyte and can accept slower output, try dense — and test both on your actual task.
Not fully on the GPU at healthy quants — that class needs substantially more memory, and crushing it to very low bits usually gives worse output than a well-quantized 30B-class model in the same footprint. Partial offload to system RAM works for patient, occasional use, but it's slow; this page only lists full-GPU fits.
Yes — the KV cache grows with context, and it grows faster on big models. The verdicts here hold at the stated context; if you plan long-context work like document analysis or coding over large files, verify your model and context in the fit calculator first.
None in particular — fit depends on memory size, so these verdicts apply to any 24 GB card. Speed is another matter: memory bandwidth varies widely between 24 GB cards, so check your specific card's GPU page for what generation speed to expect.
selected to compare · pick at least 2