LLM Torrents

Qwen

Qwen3-8B

← All models

8.2B parameters · Base · Qwen3 family

apache-2.0 hash verified source matched revision pinned

Which version should I download?

Set your rig so we can size a quant to your hardware and show which quants fit.

Set your rig

Quantizations

Format Level Size Verdict Est. speed Quality note Swarm Download
GGUF Q4_K_M 4.68 GB Set your rig — Balanced size/quality — good default. 1S / 0L webseed OK
Magnet .torrent

Run it

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m Qwen_Qwen3-8B-Q4_K_M.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./Qwen_Qwen3-8B-Q4_K_M.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
Create 
ollama create qwen-qwen3-8b -f Modelfile
Run 
ollama run qwen-qwen3-8b

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

Load Qwen_Qwen3-8B-Q4_K_M.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

Not an MLX-format quant.

MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.

.

@endif
Technical details

No template or sampling metadata recorded.

Evidence & provenance

Source

Provenance
https://huggingface.co/Qwen/Qwen3-8B
Revision pin
b968826d9c46dd6066d109eabc6255188de91218
Manifest
Present

License

Name
apache-2.0
Commercial use
yes
Access
Open

File hashes (SHA-256)

  • qwen3-8b/LICENSE 832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
  • qwen3-8b/Qwen_Qwen3-8B-Q4_K_M.gguf 54fffa050078e984116639c83dfb64b5aa6d4cd474e018b076777c632bbccccd
  • qwen3-8b/README.md 983b2abf94f46d41d17871e56ac54c4a24d6451439b8394468a472b1349f0e29

Community

Performance reports

No performance reports yet.

Reviews

No reviews yet.

Presets

No presets yet.

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard