moonshotai
Kimi-Linear-48B-A3B-Instruct
49.1B parameters · mixture-of-experts · Instruct · Kimi family
Recommended download
Which version should I download?
Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — hero-lane lead rung, DR-21. MIT.
Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.
| Format | Level | Size | Verdict | Est. speed | Quality note | Swarm | Download |
|---|---|---|---|---|---|---|---|
| GGUF | IQ4_XS | 24.65 GB | Set your rig | — | Bartowski IQ4_XS quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — hero-lane low-bit rung, DR-21. MIT. | 2S / 0L webseed OK | |
| GGUF | Q4_K_M | 28.00 GB | Set your rig | — | Bartowski Q4_K_M quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — hero-lane lead rung, DR-21. MIT. | 2S / 0L webseed OK |
GGUF · IQ4_XS
24.65 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Bartowski IQ4_XS quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — hero-lane low-bit rung, DR-21. MIT.
GGUF · Q4_K_M
28.00 GB
- Est. speed
- —
- Swarm
- 2S / 0L webseed OK
Bartowski Q4_K_M quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — hero-lane lead rung, DR-21. MIT.
Run it
Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –
- – Ollama — runs the GGUF directly; no Modelfile bundled
- – vision sidecar — not a vision model
Context / KV 8,192 tokens · FP16 set above the quant table
Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.
llama-server -m moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999Use llama-cli in place of llama-server for a one-shot prompt.
FROM ./moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999ollama create moonshotai-kimi-linear-48b-a3b-instruct -f Modelfileollama run moonshotai-kimi-linear-48b-a3b-instructGGUF is not a first-class vLLM format.
vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.
Load moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).
Not an MLX-format quant.
MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.
Technical details
No template or sampling metadata recorded.
Evidence & provenance
Source
- Revision pin
-
e1df551a447157d4658b573f9a695d57658590e9 - Manifest
- Present
License
- Name
- mit
- Commercial use
- yes
- Access
- Open
File hashes (SHA-256)
-
kimi-linear-48b-a3b-instruct-iq4-xs/LICENSE
20f48681e4fb70b2787b752fd6fb887be60332731441818458ed0baf09d8264b -
kimi-linear-48b-a3b-instruct-iq4-xs/README.md
d2332c5f1c7318737c2d42c6ec09ff7f26223d1b595fd5d0e1a189ba53064dd9 -
kimi-linear-48b-a3b-instruct-iq4-xs/moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf
96e3c084c31382539f334b0f18d3a2c1f3713492cfa4d3f7aa61d1e1340da8a8 -
kimi-linear-48b-a3b-instruct-q4-k-m/LICENSE
20f48681e4fb70b2787b752fd6fb887be60332731441818458ed0baf09d8264b -
kimi-linear-48b-a3b-instruct-q4-k-m/README.md
d2332c5f1c7318737c2d42c6ec09ff7f26223d1b595fd5d0e1a189ba53064dd9 -
kimi-linear-48b-a3b-instruct-q4-k-m/moonshotai_Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf
a1a7d865370652221f937163f7e94c99e1f114861335ba4f8666606843f1620f
Performance reports
Real-world throughput reported by the community (and scraped sources).
Reviews & presets
Community runtime reports
Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.
Log in to report whether a quant loads in your runtime.
No community reports yet.