Qwen

Qwen3-Omni-30B-A3B-Instruct

Name: Qwen3-Omni-30B-A3B-Instruct
Author: Qwen

← All models

35.3B parameters · Instruct · Qwen3 family

apache-2.0 hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Q4_K_M GGUF (ggml-org, from official Qwen3-Omni-30B-A3B-Instruct) — the all-modalities hero: text+image+audio+video in one 30B-A3B MoE. Audio/vision mmproj (Q8_0+bf16) bundled. Apache-2.0.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick?

Format	Level	Size	Verdict	Est. speed	Quality note	Swarm	Download
GGUF	Q4_K_M	17.28 GB	Set your rig	—	Q4_K_M GGUF (ggml-org, from official Qwen3-Omni-30B-A3B-Instruct) — the all-modalities hero: text+image+audio+video in one 30B-A3B MoE. Audio/vision mmproj (Q8_0+bf16) bundled. Apache-2.0.	2S / 0L webseed OK	Magnet .torrent

GGUF · Q4_K_M

17.28 GB

Set your rig

Est. speed: —
Swarm: 2S / 0L webseed OK

Q4_K_M GGUF (ggml-org, from official Qwen3-Omni-30B-A3B-Instruct) — the all-modalities hero: text+image+audio+video in one 30B-A3B MoE. Audio/vision mmproj (Q8_0+bf16) bundled. Apache-2.0.

Magnet .torrent

Run it

Runtime completeness (Q4_K_M torrent): llama.cpp ✅ Ollama – vision sidecar –

– Ollama — runs the GGUF directly; no Modelfile bundled
– vision sidecar — not a vision model

Quant

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server

llama-server -m Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile)

FROM ./Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999

Create

ollama create qwen-qwen3-omni-30b-a3b-instruct -f Modelfile

Run

ollama run qwen-qwen3-omni-30b-a3b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

Load Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

Not an MLX-format quant.

MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.

Technical details

No template or sampling metadata recorded.

Evidence & provenance

Source

Provenance: https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct
Revision pin: 26291f793822fb6be9555850f06dfe95f2d7e695
Manifest: Present

License

Name: apache-2.0
Commercial use: yes
Access: Open
Notes: HF cardData tags license:other with license_name:apache-2.0 (no license_link, no LICENSE file; README body silent on restrictions). Entire Qwen3 open release is Apache-2.0 (Qwen3-Coder-30B-A3B ships Apache-2.0 LICENSE; Qwen3-ASR/TTS apache-2.0). Treated as Apache-2.0 -> redistributable; canonical Apache-2.0 bundled in payload. Verified 2026-07-12.

File hashes (SHA-256)

qwen3-omni-30b-a3b-instruct-q4-k-m/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
qwen3-omni-30b-a3b-instruct-q4-k-m/Qwen3-Omni-30B-A3B-Instruct-Q4_K_M.gguf d9e2876556e7873e02c0359f832432ee2d67ab7dd0cee3efe0f77fd7a1f4dd85
qwen3-omni-30b-a3b-instruct-q4-k-m/README.md 94895fb5b2152384e8ba6f693d1a09b7a3adb3b0b385d33d5e363c41830d24d4
qwen3-omni-30b-a3b-instruct-q4-k-m/mmproj-Qwen3-Omni-30B-A3B-Instruct-Q8_0.gguf 1104376db833f1e89c84834144ac3863340c2cd1ddaeddb39cb0247fb5c20c8d
qwen3-omni-30b-a3b-instruct-q4-k-m/mmproj-Qwen3-Omni-30B-A3B-Instruct-bf16.gguf f0dfe825fb692d426362b1ac79678fc08daa4758f7151526cad110515f122883

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Reviews & presets

Reviews (0)

No reviews yet.

Sampler presets

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

No community reports yet.

Recommended download