Qwen

Qwen2.5-VL-32B-Instruct

← All models

33.5B parameters · Instruct · Qwen2.5 family

apache-2.0 hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — modality-lane lead rung; bundled mmproj (bf16+f16) required for vision. DR-21. Apache-2.0.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick?

GGUF · IQ4_XS

16.48 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski IQ4_XS quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — modality-lane low-bit rung; bundled mmproj (bf16+f16) required for vision. DR-21. Apache-2.0.

Magnet .torrent

GGUF · Q4_K_M

18.49 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q4_K_M quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — modality-lane lead rung; bundled mmproj (bf16+f16) required for vision. DR-21. Apache-2.0.

Magnet .torrent

GGUF · Q8_0

32.43 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q8_0 quant of Qwen2.5-VL-32B-Instruct (33.453B vision-language) — modality-lane high-quality rung; bundled mmproj (bf16+f16) required for vision. DR-21. Apache-2.0.

Magnet .torrent

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
Create 
ollama create qwen-qwen2-5-vl-32b-instruct -f Modelfile
Run 
ollama run qwen-qwen2-5-vl-32b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

Load Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

Not an MLX-format quant.

MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.

Technical details

No template or sampling metadata recorded.

Evidence & provenance

Source

Provenance
https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct
Revision pin
7cfb30d71a1f4f49a57592323337a4a4727301da
Manifest
Present

License

Name
apache-2.0
Commercial use
yes
Access
Open

File hashes (SHA-256)

  • qwen2-5-vl-32b-instruct-iq4-xs/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-iq4-xs/Qwen_Qwen2.5-VL-32B-Instruct-IQ4_XS.gguf 5cc568114dca473878f1e6467c534c68c0de31cb65ee279e4a358e8ccd6ed1eb
  • qwen2-5-vl-32b-instruct-iq4-xs/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-iq4-xs/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-iq4-xs/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9
  • qwen2-5-vl-32b-instruct-q4-k-m/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-q4-k-m/Qwen_Qwen2.5-VL-32B-Instruct-Q4_K_M.gguf 446ce4e5e746513f12e408c7936c41a219e2661a3f462469babe2936c402b516
  • qwen2-5-vl-32b-instruct-q4-k-m/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-q4-k-m/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-q4-k-m/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9
  • qwen2-5-vl-32b-instruct-q8-0/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • qwen2-5-vl-32b-instruct-q8-0/Qwen_Qwen2.5-VL-32B-Instruct-Q8_0.gguf eda114feaa726d2bd43de23460de28d2f704f45a033f5d4021c2ee7e28935e82
  • qwen2-5-vl-32b-instruct-q8-0/README.md 5f60e0526b1bc2d2365c2287f17a68153c32c092b6f0edbfd965a9ee2266b90e
  • qwen2-5-vl-32b-instruct-q8-0/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-bf16.gguf ae31b1160f891180557e0ad6c6ac4875d07a074af88d2c4f438fbd8ccefbbca7
  • qwen2-5-vl-32b-instruct-q8-0/mmproj-Qwen_Qwen2.5-VL-32B-Instruct-f16.gguf 8b07ad34435e512d4d0467d12dcbec49cbebc378e18c02b26c90fa41d8a5c7b9

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Reviews & presets

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()