moonshotai

Kimi-Linear-48B-A3B-Instruct

49.1B parameters · mixture-of-experts · Instruct · Kimi family

Mixture-of-experts — what it means for memory & speed →
mit hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Bartowski Q4_K_M quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — recommended balanced pick. MIT.

We don't have architecture data for this model, so we can't estimate whether it fits your hardware.

Quantizations

Why these quants? Which quant should I pick? What do these mean?

GGUF · IQ4_XS

24.65 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski IQ4_XS quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — compact low-bit option. MIT.

Magnet .torrent

GGUF · Q4_K_M

28.00 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Bartowski Q4_K_M quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — recommended balanced pick. MIT.

Magnet .torrent

GGUF · Q8_0

48.66 GB

Set your rig
Est. speed
—
Swarm
1S / 0L webseed OK

Bartowski Q8_0 quant of Kimi-Linear-48B-A3B-Instruct (49.123B-A3B MoE) — high-quality option. MIT. 2-part split GGUF (load via -00001-of-00002).

Magnet .torrent

Want a different quant? Request it on the board →

Context

Advertised 1M · usable ≈ 64–128k (measured — model card)

How we know

Evidence grade: measured — a published evaluation we can cite, with task and length stated.

Model card (citing Kimi Linear tech report, arXiv 2510.26692): RULER at 128k = 84.3. The 1M-token claims in the card are efficiency figures (TPOT speedups), not long-context accuracy. Advertised here = card max context (catalog field unset).

Reviewed on Jul 18, 2026.

Capabilities (as declared by the maintainer): tool calling

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands. We don't have architecture data for this model, so we can't estimate whether it fits your hardware.

Start the server 
llama-server -m moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
PARAMETER stop "[EOS]"
PARAMETER stop "<|im_end|>"
# Chat template: Ollama uses the template embedded in the GGUF (no TEMPLATE directive needed).
Create 
ollama create moonshotai-kimi-linear-48b-a3b-instruct -f Modelfile
Run 
ollama run moonshotai-kimi-linear-48b-a3b-instruct

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

transformers does not load GGUF weights.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Load moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

MLX runs MLX-format weights only (Apple Silicon). This quant is a GGUF build.

GGUF weights run under llama.cpp, Ollama or LM Studio.

Technical details

Chat template

{% macro render_content(msg) -%}
    {%- set c = msg.get('content') -%}
    {%- if c is string -%}
      {{ c }}
    {%- elif c is not none -%}
      {% for content in c -%}
        {% if content['type'] == 'image' or 'image' in content or 'image_url' in content -%}
          <|media_start|>image<|media_content|><|media_pad|><|media_end|>
        {% else -%}
          {{ content['text'] }}
        {%- endif -%}
      {%- endfor -%}
    {%- endif -%}
{%- endmacro %}


{%- if tools -%}
  <|im_system|>tool_declare<|im_middle|>{{ tools | tojson(separators=(',', ':')) }}<|im_end|>
{%- endif -%}
{% for message in messages %}  
  {%- set role_name =  message.get('name') or  message['role'] -%}
  {%- if message['role'] == 'user' -%}
    <|im_user|>{{role_name}}<|im_middle|>
  {%- elif message['role'] == 'assistant' -%}
    <|im_assistant|>{{role_name}}<|im_middle|>
  {%- else -%}
    <|im_system|>{{role_name}}<|im_middle|>
  {%- endif -%}

  {%- if message['role'] == 'assistant' and message.get('tool_calls') -%}
    {{render_content(message)}}<|tool_calls_section_begin|>
    {%- for tool_call in message['tool_calls'] -%}
        {%- set formatted_id = tool_call['id'] -%}
      <|tool_call_begin|>{{ formatted_id }}<|tool_call_argument_begin|>{% if tool_call['function']['arguments'] is string %}{{ tool_call['function']['arguments'] }}{% else %}{{ tool_call['function']['arguments'] | tojson }}{% endif %}<|tool_call_end|>
    {%- endfor -%}
    <|tool_calls_section_end|>
  {%- elif message['role'] == 'tool' -%}
    {%- set tool_call_id = message.tool_call_id -%}
    ## Return of {{ tool_call_id }}
{{render_content(message)}}
  {%- elif message['content'] is not none -%}
    {{render_content(message)}}
  {%- endif -%}
  <|im_end|>
{%- endfor -%}
{%- if add_generation_prompt -%}
  <|im_assistant|>assistant<|im_middle|>
{%- endif -%}

Stop strings

[EOS] <|im_end|>

Evidence & provenance

Source

Provenance
https://huggingface.co/moonshotai/Kimi-Linear-48B-A3B-Instruct
Revision pin
e1df551a447157d4658b573f9a695d57658590e9
Manifest
Present

License

Name
mit
Commercial use
yes
Access
Open

How verification works →

File hashes (SHA-256)

  • kimi-linear-48b-a3b-instruct-iq4-xs/LICENSE 20f48681e4fb70b2787b752fd6fb887be60332731441818458ed0baf09d8264b
  • kimi-linear-48b-a3b-instruct-iq4-xs/README.md d2332c5f1c7318737c2d42c6ec09ff7f26223d1b595fd5d0e1a189ba53064dd9
  • kimi-linear-48b-a3b-instruct-iq4-xs/moonshotai_Kimi-Linear-48B-A3B-Instruct-IQ4_XS.gguf 96e3c084c31382539f334b0f18d3a2c1f3713492cfa4d3f7aa61d1e1340da8a8
  • kimi-linear-48b-a3b-instruct-q4-k-m/LICENSE 20f48681e4fb70b2787b752fd6fb887be60332731441818458ed0baf09d8264b
  • kimi-linear-48b-a3b-instruct-q4-k-m/README.md d2332c5f1c7318737c2d42c6ec09ff7f26223d1b595fd5d0e1a189ba53064dd9
  • kimi-linear-48b-a3b-instruct-q4-k-m/moonshotai_Kimi-Linear-48B-A3B-Instruct-Q4_K_M.gguf a1a7d865370652221f937163f7e94c99e1f114861335ba4f8666606843f1620f
  • kimi-linear-48b-a3b-instruct-q8-0/LICENSE 20f48681e4fb70b2787b752fd6fb887be60332731441818458ed0baf09d8264b
  • kimi-linear-48b-a3b-instruct-q8-0/README.md d2332c5f1c7318737c2d42c6ec09ff7f26223d1b595fd5d0e1a189ba53064dd9
  • kimi-linear-48b-a3b-instruct-q8-0/moonshotai_Kimi-Linear-48B-A3B-Instruct-Q8_0-00001-of-00002.gguf 8b9902c6072803a56f89ab515c363d433117b2ba601989565a3aa9c0836771bc
  • kimi-linear-48b-a3b-instruct-q8-0/moonshotai_Kimi-Linear-48B-A3B-Instruct-Q8_0-00002-of-00002.gguf 999d77a85f36d296c557e26a4a0ced2c628c6f1faed79ca3550442d7c02a45c5

Explore further

Also consider

Get an email when a better quant fits your rig.

Set your rig once; we'll alert you when a new or better-fitting build lands.

Set your rig

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard
Community

Reviews, sampler presets and community runtime reports. Be the first to contribute.

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()