On a 24 GB GPU, 2 catalog models run fully on the GPU at an 8,192-token context. The most capable is Qwen3-Coder-30B-A3B-Instruct at Q4_K_M (needs ~20.3 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 24 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
24 GB runs the current crop of dedicated code models fully on the GPU at 4-bit sweet-spot quants. This page lists the catalog's code-specialist models that fit; it's a short, focused list by design — general instruct models handle code too, and the full 24 GB list shows everything the card runs. For coding specifically, context headroom matters as much as model choice.
Coding work is long-context work: repository files, diffs, error logs, and multi-turn sessions all pile into the context window, and the KV cache that holds it grows accordingly. A code model that fits at this page's stated context may need a tighter quant — or a shorter window — once you feed it real project files. Check your intended context in the calculator before settling on a quant.
Qwen3-Coder-30B-A3B-Instruct at Q4_K_M · 30.5B params · needs ~20.3 GiB
One command to run it (llama.cpp):
llama-server -m Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| Qwen3-Coder-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Devstral-Small-2-24B-Instruct-2512 mistralai | Q4_K_M Runs fully on GPU | ~16.5 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Because it only lists models the catalog tags as code specialists that also run fully on a 24 GB GPU. Plenty of general instruct models are competent at code — if you want the full menu of what 24 GB runs, see the general 24 GB page and test a general model against a specialist on your own tasks.
Specialists are trained heavily on code and tend to be stronger at completion, editing, and tool-driven coding workflows; strong general models are often close behind and more versatile for mixed work. If coding is most of your usage, start with a specialist; otherwise a general model of similar size is a reasonable default.
More than for chat — files, diffs, and logs add up quickly, and agent-style workflows compound it. The KV cache grows with context, so budget memory for the window you'll actually use, not the default: the fit calculator lets you check your exact model, quant, and context together.
Yes. Run the model with llama.cpp's server (the command on this page) or a desktop runtime like LM Studio or Ollama, and point any editor plugin or coding agent that supports a custom OpenAI-compatible endpoint at it. No cloud account required — that's the point of running locally.
Mixture-of-experts code models generate faster per token than dense models of similar total size, which matters in interactive editing and agent loops that make many calls. Dense models can be stronger per gigabyte on hard problems. If your workflow is interactive, speed usually wins — test both on your own codebase.
selected to compare · pick at least 2