tencent

Hunyuan-A13B-Instruct

← All models

80.4B parameters · mixture-of-experts · Instruct · Hunyuan family

other hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — hero-lane lead rung, DR-21. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick?

GGUF · IQ4_XS

40.50 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski IQ4_XS quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — hero-lane low-bit rung, DR-21. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Magnet .torrent

GGUF · Q4_K_M

45.93 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q4_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — hero-lane lead rung, DR-21. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Magnet .torrent

GGUF · Q5_K_M

53.64 GB

Set your rig
Est. speed
—
Swarm
—

Bartowski Q5_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — hero-lane quality rung, DR-21. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea). 2-part split GGUF (load via -00001-of-00002).

Not yet available

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
Create 
ollama create tencent-hunyuan-a13b-instruct -f Modelfile
Run 
ollama run tencent-hunyuan-a13b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

Load tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

Not an MLX-format quant.

MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.

Technical details

No template or sampling metadata recorded.

Evidence & provenance

Source

Provenance
https://huggingface.co/tencent/Hunyuan-A13B-Instruct
Revision pin
290ddb9a56ed23c2c83a1c8081533e58925df952
Manifest
Present

License

Name
other
Commercial use
unclear
Access
Open

File hashes (SHA-256)

  • hunyuan-a13b-instruct-iq4-xs/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
  • hunyuan-a13b-instruct-iq4-xs/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
  • hunyuan-a13b-instruct-iq4-xs/tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf 54d09caf5a7f92b594b49c35a37582755a804c289c8c09e78f350499730676d7
  • hunyuan-a13b-instruct-q4-k-m/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
  • hunyuan-a13b-instruct-q4-k-m/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
  • hunyuan-a13b-instruct-q4-k-m/tencent_Hunyuan-A13B-Instruct-Q4_K_M.gguf 81a8c03ffcac5ba3a919a956bfb8f068891d68784fe93ac14b938fac4ef4038f

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Reviews & presets

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()