HuggingFaceTB

SmolLM3-3B

← All models

3.1B parameters · Instruct · SmolLM3 family

apache-2.0 hash verified source matched revision pinned

Which version should I download?

Default pick: GGUF · Q4_K_M — Unsloth Q4_K_M quant of SmolLM3-3B (3.075B dense) — compact edge/CPU lead rung, DR-21. Apache-2.0.

Set your rig to confirm it fits and to see if a higher-quality quant runs fully on your hardware.

Set your rig

Quantizations

Why these quants? Which quant should I pick?

GGUF · IQ4_XS

1.61 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Unsloth IQ4_XS quant of SmolLM3-3B (3.075B dense) — compact edge/CPU low-bit rung, DR-21. Apache-2.0.

Magnet .torrent

GGUF · Q4_K_M

1.78 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Unsloth Q4_K_M quant of SmolLM3-3B (3.075B dense) — compact edge/CPU lead rung, DR-21. Apache-2.0.

Magnet .torrent

GGUF · Q8_0

3.05 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Unsloth Q8_0 quant of SmolLM3-3B (3.075B dense) — high-quality rung, DR-21. Apache-2.0.

Magnet .torrent

GGUF · BF16

5.74 GB

Set your rig
Est. speed
—
Swarm
2S / 0L webseed OK

Unsloth BF16 GGUF of SmolLM3-3B (3.075B dense) — source-quality preservation rung (small model). Apache-2.0.

Magnet .torrent

Run it

Runtime completeness (IQ4_XS torrent): llama.cpp ✅ Ollama – vision sidecar –

  • – Ollama — runs the GGUF directly; no Modelfile bundled
  • – vision sidecar — not a vision model

Context / KV 8,192 tokens · FP16 set above the quant table

Generic commands (no rig set). GPU-offload values assume the model fits on your GPU — set your rig for values tuned to your hardware.

Start the server 
llama-server -m SmolLM3-3B-IQ4_XS.gguf -c 8192 -ngl 999

Use llama-cli in place of llama-server for a one-shot prompt.

Save as Modelfile next to the GGUF (Modelfile) 
FROM ./SmolLM3-3B-IQ4_XS.gguf
PARAMETER num_ctx 8192
PARAMETER num_gpu 999
Create 
ollama create huggingfacetb-smollm3-3b -f Modelfile
Run 
ollama run huggingfacetb-smollm3-3b

GGUF is not a first-class vLLM format.

vLLM GGUF support is experimental and single-file only; prefer safetensors/GPTQ/AWQ for production. If you must, pass the .gguf path to `vllm serve` with --load-format gguf on a recent vLLM.

Load SmolLM3-3B-IQ4_XS.gguf, set the context length to 8192 tokens. Set GPU offload to Max (all layers).

Not an MLX-format quant.

MLX runs MLX-format weights only (Apple Silicon). This quant is not an MLX build.

Technical details

No template or sampling metadata recorded.

Evidence & provenance

Source

Provenance
https://huggingface.co/HuggingFaceTB/SmolLM3-3B
Revision pin
a07cc9a04f16550a088caea529712d1d335b0ac1
Manifest
Present

License

Name
apache-2.0
Commercial use
yes
Access
Open

File hashes (SHA-256)

  • smollm3-3b-iq4-xs/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • smollm3-3b-iq4-xs/README.md cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8
  • smollm3-3b-iq4-xs/SmolLM3-3B-IQ4_XS.gguf c1ab400f2c4375d3c949af0d6ea4ef4ac71691430bb8b9d92af887a400ee7306
  • smollm3-3b-q4-k-m/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • smollm3-3b-q4-k-m/README.md cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8
  • smollm3-3b-q4-k-m/SmolLM3-3B-Q4_K_M.gguf 4de907d2d388a5508fb7cb443a06effe14cce3518b0a78d3bdd9e74d9edce989
  • smollm3-3b-q8-0/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • smollm3-3b-q8-0/README.md cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8
  • smollm3-3b-q8-0/SmolLM3-3B-Q8_0.gguf 5bd828e684d883459a03ce1eec9dc9044345c6da64075d9ae7e890dedc68de28
  • smollm3-3b-bf16/LICENSE cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30
  • smollm3-3b-bf16/README.md cc3e2985fe1b789e2c7b9be5fe0d226d09e1e52bf030fe049276bf9d64a571b8
  • smollm3-3b-bf16/SmolLM3-3B-BF16.gguf 3082a0be39c4014306e8aa688b2f8051468fe077fc99835ad576e21e83f93788

Performance reports

Real-world throughput reported by the community (and scraped sources).

View performance leaderboard

Reviews & presets

Reviews (0)

Log in to write a structured review of this model.

No reviews yet.

Sampler presets

Log in to share a sampler preset or vote on presets.

No presets yet.

Community runtime reports

Unofficial, community-submitted "it loads for me" reports. These are not the official verified-working badge — an admin reviews reports before anything is marked officially verified.

Log in to report whether a quant loads in your runtime.

No community reports yet.

selected to compare · pick at least 2

Compare ()