deepseek-ai

DeepSeek-V4-Flash

Name: DeepSeek-V4-Flash
Author: deepseek-ai

290.9B parameters · mixture-of-experts · Base · DeepSeek family

Mixture-of-experts — what it means for memory & speed → Long context — how it uses memory →

mit hash verified source matched revision pinned

Good for: Long context

Which version should I download?

Default pick: GGUF · UD-Q2_K_XL — Unsloth dynamic 2-bit — runnable giant lead (fits ~128 GB RAM w/ MoE offload).

We don't have architecture data for this model, so we can't estimate whether it fits your hardware.

Quantizations

Why these quants? Which quant should I pick? What do these mean?

Format	Level	Size	Verdict	Est. speed	Quality note	Swarm	Download
GGUF	UD-Q2_K_XL	90.18 GB	Set your rig	—	Unsloth dynamic 2-bit — runnable giant lead (fits ~128 GB RAM w/ MoE offload).	1S / 0L webseed OK	Magnet .torrent

GGUF · UD-Q2_K_XL

90.18 GB

Set your rig

Est. speed: —
Swarm: 1S / 0L webseed OK

Unsloth dynamic 2-bit — runnable giant lead (fits ~128 GB RAM w/ MoE offload).

Magnet .torrent

Want a different quant? Request it on the board →

Context

Advertised 1M · usable ≈ 256k–1M (measured — model card)

How we know

Evidence grade: measured — a published evaluation we can cite, with task and length stated.

DeepSeek-V4-Flash model card — observed Jul 18, 2026
HF blog: DeepSeek-V4 long context — observed Jul 18, 2026

Model card: MRCR at 1M 78.7 (Flash Max), CorpusQA at 1M 60.5, LongBench-V2 51.5; card recommends >=384K window for Think Max. HF blog cautions capacity is not performance (its stronger through-256k figure is V4-Pro-Max, not Flash).

Reviewed on Jul 18, 2026.

Run it

Runtime completeness (UD-Q2_K_XL torrent): llama.cpp ✅ Ollama – vision sidecar –

– Ollama — runs the GGUF directly; no Modelfile bundled
– vision sidecar — not a vision model

Quant

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands. We don't have architecture data for this model, so we can't estimate whether it fits your hardware.

Start the server

llama-server -m DeepSeek-V4-Flash-UD-Q2_K_XL-00001-of-00003.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile)

FROM ./DeepSeek-V4-Flash-UD-Q2_K_XL-00001-of-00003.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER temperature 1
PARAMETER top_p 1
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).

Create

ollama create deepseek-ai-deepseek-v4-flash -f Modelfile

Run

ollama run deepseek-ai-deepseek-v4-flash

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load DeepSeek-V4-Flash-UD-Q2_K_XL-00001-of-00003.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Sampling defaults

top_p: 1
temperature: 1

Stop strings

<｜end▁of▁sentence｜>

Evidence & provenance

Source

Provenance: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash
Revision pin: 60d8d70770c6776ff598c94bb586a859a38244f1
Manifest: Present

License

Name: mit
Commercial use: yes
Access: Open

How verification works →

File hashes (SHA-256)

deepseek-v4-flash-ud-q2-k-xl/DeepSeek-V4-Flash-UD-Q2_K_XL-00001-of-00003.gguf 5475dce5d7189b4304d490bf559d438c967bbc924ce5384ba8e9ecc0385cb003
deepseek-v4-flash-ud-q2-k-xl/DeepSeek-V4-Flash-UD-Q2_K_XL-00002-of-00003.gguf 3faf8797868960141239b69bdd8105b0459fbc0f1927652889ae936a967fc4ca
deepseek-v4-flash-ud-q2-k-xl/DeepSeek-V4-Flash-UD-Q2_K_XL-00003-of-00003.gguf 4083f1f78e2730006acf78a458ac6b190faf1f515c30cce479a6f8d5066dda80
deepseek-v4-flash-ud-q2-k-xl/LICENSE f2c6c602815669d292889e5be8c802f2ed950653b77999b1584e8e6aed25d040
deepseek-v4-flash-ud-q2-k-xl/README.md f437c84c5baca622db81ccc449aff9c2c5c6c4caff7c5f659bc0980198508b21

Explore further

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

No reviews yet.

Sampler presets

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

No community reports yet.

Recommended download