Qwen

Qwen3-32B

Name: Qwen3-32B
Author: Qwen

32.8B parameters · reasoning · Base · Qwen3 family

Long context — how it uses memory →

apache-2.0 hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Balanced size/quality — good default.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

Format	Level	Size	Verdict	Est. speed	Quality note	Swarm	Download
GGUF	IQ4_XS	16.48 GB	Set your rig	—	Bartowski IQ4_XS quant of Qwen3-32B (32.762B dense) — compact low-bit option. Apache-2.0.	2S / 0L webseed OK	Magnet .torrent
GGUF	Q4_K_M	18.40 GB	Set your rig	—	Balanced size/quality — good default.	2S / 0L webseed OK	Magnet .torrent
GGUF	Q8_0	32.43 GB	Set your rig	—	Bartowski Q8_0 quant of Qwen3-32B (32.762B dense) — high-quality option. Apache-2.0.	2S / 0L webseed OK	Magnet .torrent

GGUF · IQ4_XS

16.48 GB

Set your rig

Est. speed: —
Swarm: 2S / 0L webseed OK

Bartowski IQ4_XS quant of Qwen3-32B (32.762B dense) — compact low-bit option. Apache-2.0.

Magnet .torrent

GGUF · Q4_K_M

18.40 GB

Set your rig

Est. speed: —
Swarm: 2S / 0L webseed OK

Balanced size/quality — good default.

Magnet .torrent

GGUF · Q8_0

32.43 GB

Set your rig

Est. speed: —
Swarm: 2S / 0L webseed OK

Bartowski Q8_0 quant of Qwen3-32B (32.762B dense) — high-quality option. Apache-2.0.

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

RTX 5090 · Q4_K_M RTX 4090 · Q4_K_M RTX 3090 · Q4_K_M RTX 3090 Ti · Q4_K_M RTX Pro 6000 Blackwell 96GB · Q8_0

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 40k · usable ≈ 64–128k (measured — RULER)

How we know

Evidence grade: measured — a published evaluation we can cite, with task and length stated.

NVIDIA RULER leaderboard — observed Jul 18, 2026
Qwen3-32B model card — observed Jul 18, 2026

RULER (author-reported): 91.8 at 64k, 85.6 at 128k; effective length >128k per leaderboard. Card: native 32,768, 131,072 via YaRN.

Reviewed on Jul 18, 2026.

Capabilities (as declared by the maintainer): reasoning tool calling

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

– Ollama — runs the GGUF directly; no Modelfile bundled
– vision sidecar — not a vision model

Quant

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server

llama-server -m Qwen_Qwen3-32B-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile)

FROM ./Qwen_Qwen3-32B-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).

Create

ollama create qwen-qwen3-32b -f Modelfile

Run

ollama run qwen-qwen3-32b

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load Qwen_Qwen3-32B-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
{%- for message in messages[::-1] %}
    {%- set index = (messages|length - 1) - loop.index0 %}
    {%- if ns.multi_step_tool and message.role == "user" and message.content is string and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}
        {%- set ns.multi_step_tool = false %}
        {%- set ns.last_query_index = index %}
    {%- endif %}
{%- endfor %}
{%- for message in messages %}
    {%- if message.content is string %}
        {%- set content = message.content %}
    {%- else %}
        {%- set content = '' %}
    {%- endif %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is string %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '</think>' in content %}
                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content.strip('\n') + '\n</think>\n\n' + content.lstrip('\n') }}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {%- if (loop.first and content) or (not loop.first) %}
                    {{- '\n' }}
                {%- endif %}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' }}
                {{- tool_call.name }}
                {{- '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking is false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}

Sampling defaults

top_k: 20
top_p: 0.95
temperature: 0.6

Stop strings

<|im_end|> <|endoftext|>

Evidence & provenance

Source

Provenance: https://huggingface.co/Qwen/Qwen3-32B
Revision pin: 9216db5781bf21249d130ec9da846c4624c16137
Manifest: Present

License

Name: apache-2.0
Commercial use: yes
Access: Open

How verification works →

File hashes (SHA-256)

qwen3-32b-iq4-xs/LICENSE 832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
qwen3-32b-iq4-xs/Qwen_Qwen3-32B-IQ4_XS.gguf e1e49f04bab5c5aa55d15c7cd383f01672cb0777c2d0153587bfd1a2230ea8ec
qwen3-32b-iq4-xs/README.md 461cff52459624025d46c4f67143035cffd2a4cacd75bcb3c66eae8bf3a13553
qwen3-32b/LICENSE 832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
qwen3-32b/Qwen_Qwen3-32B-Q4_K_M.gguf e41ec56ddd376963a116da97506fadfccb50fb402bb6f3cb4be0bc179a582bd6
qwen3-32b/README.md 461cff52459624025d46c4f67143035cffd2a4cacd75bcb3c66eae8bf3a13553
qwen3-32b-q8-0/LICENSE 832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
qwen3-32b-q8-0/Qwen_Qwen3-32B-Q8_0.gguf bcb4c2d84e1d413fd3239ddaff64d5cca0928fe84af2349dee5f0ee3cc8ed84a
qwen3-32b-q8-0/README.md 461cff52459624025d46c4f67143035cffd2a4cacd75bcb3c66eae8bf3a13553

Explore further

Appears in

What runs on 24 GB of VRAM What runs on 32 GB of VRAM What runs on 48 GB of VRAM What runs on a 36 GB Mac What runs on a 48 GB Mac What runs on a 64 GB Mac

Also consider

Local alternative to

A local alternative to Claude Opus → A local alternative to Claude Sonnet → A local alternative to GPT (latest) → A local alternative to Gemini (latest) →

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

No reviews yet.

Sampler presets

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

No community reports yet.

Recommended download

Quantizations

Runs fully on

Context

Run it

Chat template

Sampling defaults

Stop strings

Evidence & provenance

Source

License

File hashes (SHA-256)

Explore further

Appears in

Also consider

Local alternative to

Performance reports

Reviews (0)

Sampler presets

Community runtime reports