deepseek-ai
DeepSeek-V3-0324
684.5B parameters · mixture-of-experts · Base · DeepSeek family
Recommended download
Which version should I download?
Default pick: GGUF · UD-Q2_K_XL — Unsloth UD-Q2_K_XL dynamic 2-bit quant of DeepSeek-V3-0324 (684.531B total, 256-expert MoE) — DR-21 giants-lane lead rung. MIT.
Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.
| Format | Level | Size | Verdict | Est. speed | Quality note | Swarm | Download |
|---|---|---|---|---|---|---|---|
| GGUF | UD-Q2_K_XL | 230.59 GB | Set your rig | — | Unsloth UD-Q2_K_XL dynamic 2-bit quant of DeepSeek-V3-0324 (684.531B total, 256-expert MoE) — DR-21 giants-lane lead rung. MIT. | 2S / 0L webseed OK |
GGUF · UD-Q2_K_XL
230.59 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Unsloth UD-Q2_K_XL dynamic 2-bit quant of DeepSeek-V3-0324 (684.531B total, 256-expert MoE) — DR-21 giants-lane lead rung. MIT.
Run it
Runtime completeness (UD-Q2_K_XL torrent): llama.cpp ✅ Ollama – vision sidecar –
- – Ollama — runs the GGUF directly; no Modelfile bundled
- – vision sidecar — not a vision model
Context / KV 8,192 tokens · FP16 set above the quant table
Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.
llama-server -m DeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf -c 8192 -ngl 999Use llama-cli in place of llama-server for a one-shot prompt.
FROM ./DeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999ollama create deepseek-ai-deepseek-v3-0324 -f Modelfileollama run deepseek-ai-deepseek-v3-0324GGUF is not a first-class vLLM format.
vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.
Load DeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).
Not an MLX-format quant.
MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.
Technical details
No template or sampling metadata recorded.
Evidence & provenance
Source
- Revision pin
-
e9b33add76883f293d6bf61f6bd89b497e80e335 - Manifest
- Present
License
- Name
- mit
- Commercial use
- yes
- Access
- Open
File hashes (SHA-256)
-
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00001-of-00006.gguf
a175fb9082806ffae03c890b9a2daae46d24f39b616e0e9d9908f9b212e0ab76 -
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00002-of-00006.gguf
2aa680995be7c46d423515477261fabdda9cbca7cbc60bba877a58143fdb7582 -
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00003-of-00006.gguf
66fcda6a99d901b5fa4c5ff0eabff57af035d09ebb9f4e6ed41ad6926ef9cd0d -
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00004-of-00006.gguf
63ac78c8363784885a985beff75e375c6bb63f6b31ff905b2ac5df0267f3e389 -
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00005-of-00006.gguf
d3e6cdee637bd94744c971ffb8df34064a8a26ac0fc2e60e8321b427373a8ca6 -
deepseek-v3-0324-ud-q2-k-xl/DeepSeek-V3-0324-UD-Q2_K_XL-00006-of-00006.gguf
28c72c67d74403ea758faf93217672b0a14d2bb32d21b4bf1d53846ae7f8b536 -
deepseek-v3-0324-ud-q2-k-xl/LICENSE
1c8f573e830ca9b3ebfeb7ace1823146e22b66f99ee223840e7637c9e745e1c7 -
deepseek-v3-0324-ud-q2-k-xl/README.md
6c0954d00496da52100a57fbc15101a96a1bf18099e973e21817d95eaa5870e8
Performance reports
Real-world throughput reported by the community (and scraped sources).
Reviews & presets
Community runtime reports
Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.
Log in to report whether a quant loads in your runtime.
No community reports yet.