tencent

Hunyuan-A13B-Instruct

Name: Hunyuan-A13B-Instruct
Author: tencent

80.4B parameters · mixture-of-experts · Instruct · Hunyuan family

Mixture-of-experts — what it means for memory & speed → Long context — how it uses memory →

other hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — recommended balanced pick. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick? What do these mean?

Format	Level	Size	Verdict	Est. speed	Quality note	Swarm	Download
GGUF	IQ4_XS	40.50 GB	Set your rig	—	Bartowski IQ4_XS quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — compact low-bit option. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).	2S / 0L webseed OK	Magnet .torrent
GGUF	Q4_K_M	45.93 GB	Set your rig	—	Bartowski Q4_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — recommended balanced pick. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).	1S / 0L webseed OK	Magnet .torrent
GGUF	Q5_K_M	53.64 GB	Set your rig	—	Bartowski Q5_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — quality step-up. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea). 2-part split GGUF (load via -00001-of-00002).	1S / 0L webseed OK	Magnet .torrent
GGUF	Q8_0	79.58 GB	Set your rig	—	Bartowski Q8_0 quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — high-quality option. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea). 3-part split GGUF (load via -00001-of-00003).	1S / 0L webseed OK	Magnet .torrent

GGUF · IQ4_XS

40.50 GB

Set your rig

Est. speed: —
Swarm: 2S / 0L webseed OK

Bartowski IQ4_XS quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — compact low-bit option. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Magnet .torrent

GGUF · Q4_K_M

45.93 GB

Set your rig

Est. speed: —
Swarm: 1S / 0L webseed OK

Bartowski Q4_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — recommended balanced pick. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea).

Magnet .torrent

GGUF · Q5_K_M

53.64 GB

Set your rig

Est. speed: —
Swarm: 1S / 0L webseed OK

Bartowski Q5_K_M quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — quality step-up. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea). 2-part split GGUF (load via -00001-of-00002).

Magnet .torrent

GGUF · Q8_0

79.58 GB

Set your rig

Est. speed: —
Swarm: 1S / 0L webseed OK

Bartowski Q8_0 quant of Hunyuan-A13B-Instruct (80.393B-A13B MoE) — high-quality option. Tencent Hunyuan Community License (territory-restricted: excludes EU/UK/South Korea). 3-part split GGUF (load via -00001-of-00003).

Magnet .torrent

Want a different quant? Request it on the board →

Runs fully on

RTX Pro 6000 Blackwell 96GB · Q8_0 RTX A6000 · IQ4_XS Apple M4 Max (128 GB) · Q8_0 Apple M4 Max (64 GB) · IQ4_XS Apple M4 Pro (64 GB) · IQ4_XS

Green = the model's best quant fits fully in GPU/unified memory at 8K context. Tap a card for its full "what runs on it" page.

Context

Advertised 32k · usable ≈ 256k (reported — model card)

How we know

Evidence grade: reported — a maintainer claim with no independent evaluation.

Hunyuan-A13B-Instruct model card — observed Jul 18, 2026
Hunyuan-A13B GitHub README — observed Jul 18, 2026

Card: supports up to 256K (262,144), but the shipped config.json limits context to 32K as an OOM guard — the catalog advertised value reflects the shipped config. No public long-context scores (tech-report PDF not machine-verifiable).

Reviewed on Jul 18, 2026.

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

– Ollama — runs the GGUF directly; no Modelfile bundled
– vision sidecar — not a vision model

Quant

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server

llama-server -m tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile)

FROM ./tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "<|eos|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER repeat_penalty 1.05
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).

Create

ollama create tencent-hunyuan-a13b-instruct -f Modelfile

Run

ollama run tencent-hunyuan-a13b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{% set context = {'has_head': true} %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = message['content'] %}{% if loop.index0 == 0 %}{% if content == '' %}{% set _ = context.update({'has_head': false}) %}{% elif message['role'] == 'system' %}{% set content = '<|startoftext|>' + content + '<|extra_4|>' %}{% endif %}{% endif %}{% if message['role'] == 'user' %}{% if loop.index0 == 1 and not context.has_head %}{% set content = '<|startoftext|>' + content %}{% endif %}{% if loop.index0 == 1 and context.has_head %}{% set content = content + '<|extra_0|>' %}{% else %}{% set content = '<|startoftext|>' + content + '<|extra_0|>' %}{% endif %}{% elif message['role'] == 'assistant' %}{% set content = content + '<|eos|>' %}{% endif %}{{ content }}{% endfor %}

Sampling defaults

top_k: 20
top_p: 0.8
temperature: 0.7
repetition_penalty: 1.05

Stop strings

<|eos|>

Evidence & provenance

Source

Provenance: https://huggingface.co/tencent/Hunyuan-A13B-Instruct
Revision pin: 290ddb9a56ed23c2c83a1c8081533e58925df952
Manifest: Present

License

Name: other
Commercial use: unclear
Access: Open

How verification works →

File hashes (SHA-256)

hunyuan-a13b-instruct-iq4-xs/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
hunyuan-a13b-instruct-iq4-xs/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
hunyuan-a13b-instruct-iq4-xs/tencent_Hunyuan-A13B-Instruct-IQ4_XS.gguf 54d09caf5a7f92b594b49c35a37582755a804c289c8c09e78f350499730676d7
hunyuan-a13b-instruct-q4-k-m/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
hunyuan-a13b-instruct-q4-k-m/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
hunyuan-a13b-instruct-q4-k-m/tencent_Hunyuan-A13B-Instruct-Q4_K_M.gguf 81a8c03ffcac5ba3a919a956bfb8f068891d68784fe93ac14b938fac4ef4038f
hunyuan-a13b-instruct-q5-k-m/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
hunyuan-a13b-instruct-q5-k-m/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
hunyuan-a13b-instruct-q5-k-m/tencent_Hunyuan-A13B-Instruct-Q5_K_M-00001-of-00002.gguf fbbe6f7a80f094e3809812cb77e724040270a57895906798c1e1a6e6d6a4cf05
hunyuan-a13b-instruct-q5-k-m/tencent_Hunyuan-A13B-Instruct-Q5_K_M-00002-of-00002.gguf b6d56d7689dd82cc164581cce70f4cd1cfa961a33e6a36b8771314b9267f6123
hunyuan-a13b-instruct-q8-0/LICENSE e3e51427556222cb152990f47991eedbd6ffa6561bb9346580d32e841e13530c
hunyuan-a13b-instruct-q8-0/README.md f7df4703f678f1baa19e4786be5fd8d55559bb3dc83c84cb6e015fb1aa08a969
hunyuan-a13b-instruct-q8-0/tencent_Hunyuan-A13B-Instruct-Q8_0-00001-of-00003.gguf f75dfc97a46210edfe7a3737c8587b507aad939d3c6d8f49f7eedc1adf71dcc6
hunyuan-a13b-instruct-q8-0/tencent_Hunyuan-A13B-Instruct-Q8_0-00002-of-00003.gguf 80951ed01a22ec9759747a872be3102b320b9c53f4349adf94ef7ab0a15ffd4d
hunyuan-a13b-instruct-q8-0/tencent_Hunyuan-A13B-Instruct-Q8_0-00003-of-00003.gguf f136d19fa704d868b053048398814c11770b6ee24f3e3a01fe42afc2a9df3a22

Explore further

Appears in

What runs on 48 GB of VRAM What runs on a 64 GB Mac What runs on a 128 GB Mac What runs on a 192 GB Mac Writing models that fit 48 GB General models that fit 48 GB

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

No reviews yet.

Sampler presets

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

No community reports yet.

Recommended download

Quantizations

Runs fully on

Context

Run it

Chat template

Sampling defaults

Stop strings

Evidence & provenance

Source

License

File hashes (SHA-256)

Explore further

Appears in

Also consider

Performance reports

Reviews (0)

Sampler presets

Community runtime reports