On a 48 GB GPU, 12 catalog models run fully on the GPU at an 8,192-token context. The most capable is Hunyuan-A13B-Instruct at IQ4_XS (needs ~46.2 GiB). Pick a smaller model or a lower quant for more headroom.
Figures assume a 48 GB GPU paired with 32 GB of system RAM (a typical desktop), judged at an 8,192-token context. Speed depends on the specific card — this page ranks by fit, not speed.
TL;DR
48 GB is where the 70B class finally runs fully on the GPU. Dense models in that class fit at compact 4-bit quants, the biggest mixture-of-experts models in the catalog squeeze in, and the whole 30B class runs at near-lossless quants with room to spare. The list below is derived live from the fit engine, largest first.
Plenty of people reach this tier with two 24 GB cards rather than one workstation card. For fit, that mostly works: llama.cpp splits a model's layers across GPUs, so as a rule of thumb the combined memory is your budget, minus a little overhead per card. The verdicts on this page assume a single 48 GB pool, so treat them as close-but-not-exact for a split rig — and expect some speed cost from crossing between cards, especially in interactive use.
Hunyuan-A13B-Instruct at IQ4_XS · 80.4B params · needs ~46.2 GiB
One command to run it (llama.cpp):
llama-server -m tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999| Model | Sweet-spot quant | Fits in |
|---|---|---|
| Hunyuan-A13B-Instruct tencent | IQ4_XS Runs fully on GPU | ~46.2 GiB |
| Kimi-Dev-72B moonshotai | IQ4_XS Runs fully on GPU | ~43.8 GiB |
| Qwen3.6-35B-A3B Qwen | Q8_0 Runs fully on GPU | ~39 GiB |
| Qwen3-Omni-30B-A3B-Instruct Qwen | Q4_K_M Runs fully on GPU | ~20.3 GiB |
| Qwen2.5-VL-32B-Instruct Qwen | Q8_0 Runs fully on GPU | ~38.3 GiB |
| DeepSeek-R1-Distill-Qwen-32B deepseek-ai | Q8_0 Runs fully on GPU | ~38.3 GiB |
| Qwen3-32B Qwen | Q8_0 Runs fully on GPU | ~38.3 GiB |
| gemma-4-31B-it google | Q8_0 Runs fully on GPU | ~36 GiB |
| GLM-4.7-Flash zai-org | Q8_0 Runs fully on GPU | ~33.6 GiB |
| Qwen3-Coder-30B-A3B-Instruct Qwen | Q8_0 Runs fully on GPU | ~34.6 GiB |
| Qwen3-30B-A3B-Instruct-2507 Qwen | Q8_0 Runs fully on GPU | ~34.6 GiB |
| Qwen3.6-27B Qwen | Q8_0 Runs fully on GPU | ~31.9 GiB |
Derived live from the fit engine + catalog at an 8,192-token context. "Fits in" is the modelled VRAM the sweet-spot quant needs (weights + KV cache + overhead). Speed depends on your specific card — check a GPU page or the calculator.
Yes — fully on the GPU at compact 4-bit quants, at moderate context. That's the headline this tier earns. Near-lossless quants of that class are beyond 48 GB, and long contexts eat the remaining margin quickly, so check your intended window in the fit calculator before committing to a very large download.
For fit, roughly — llama.cpp can split a model's layers across the cards, so the combined memory is what mostly matters, minus a little per-card overhead. Speed is the difference: moving between cards costs something, and two cards rarely match one card of the same total memory. The verdicts here assume a single pool; treat them as a close guide for a split rig, not a guarantee.
For raw capability, usually — parameter count generally beats bit-depth at these scales. But the 30B-class option generates faster, leaves generous room for context, and at near-lossless quants tracks its original model closely. If your work is interactive or long-context, the smaller model often wins in practice; test both on your task.
Yes, and it bites hardest here — the KV cache grows with both context and model size, so the 70B-class entries that barely fit at this page's context are the first to stop fitting at longer ones. Verify your model, quant, and target context together in the fit calculator.
This page ranks by fit, not speed. 48 GB setups span workstation cards, older server cards, and dual-card rigs, and their memory bandwidth differs widely — so identical fit verdicts can come with very different generation speeds. Check your specific card's GPU page for speed context.
selected to compare · pick at least 2