K-quants, I-quants and imatrix
What the K in Q4_K_M means, how I-quants (IQ...) differ, and what an importance matrix (imatrix) actually does during quantization.
3 min read Reviewed July 2026
TL;DR
GGUF quant names hide three different ideas. K-quants (Q4_K_M and friends) are the mainstream default for 4–6-bit work. I-quants (IQ3_XS, IQ2_XXS, …) use a cleverer encoding that holds quality much better at very low bit-rates (2–3 bit), sometimes at a small decode-speed cost. An imatrix is neither — it is a build-time calibration file that helps any quant, especially low-bit ones, allocate precision more intelligently.
These three are often mixed up, so here is what each one is — and how to tell them apart from the name alone. For the wider format landscape (GGUF vs AWQ/GPTQ/EXL2/MLX), start with Quantization formats explained.
How do I read the name?
| Prefix | Family | Examples | In one line |
|---|---|---|---|
Q*_0 / Q*_1 |
Legacy quants | Q4_0, Q8_0 |
Simple fixed-size blocks, one scale per block. Q8_0 is still a fine near-lossless option; for 4–6-bit work these are largely superseded |
Q*_K (+ _S/_M/_L) |
K-quants | Q4_K_M, Q6_K |
Spend more bits on the tensors that matter most; the mainstream default |
IQ* (+ _XXS/_XS/…) |
I-quants | IQ3_XS, IQ2_XXS |
Codebook/lookup-style encoding; the low-bit specialists |
The S/M/L (and XXS/XS) suffixes are size classes within a family — a larger class spends a few more bits on the most important tensors (Which quant should I pick? covers choosing a level).
K-quants vs I-quants: what's the actual difference?
K-quants (Q4_K_M …)
- Sweet spot: 4–6 bit — the mainstream range
- Encoding: block-based, allocating more bits to the tensors that matter most
- Decode speed: fast everywhere — simple dequantization
- Use when: the model fits at 4-bit or above;
Q4_K_Mis the workhorse
I-quants (IQ3_XS …)
- Sweet spot: very low bit-rates — the 2–3-bit range
- Encoding: more sophisticated codebook/lookup-style scheme, squeezing more quality per bit
- Decode speed: can be a bit slower, especially on CPUs and older GPUs
- Use when: the model barely fits; I-quants keep quality from falling off a cliff
In prose: K-quants hold quality better per bit than the legacy quants because they spend bits unevenly, favouring the weights that matter. I-quants push that further with a lookup-style encoding that pays off most where every bit is precious — at 2–3 bits they hold up noticeably better than any K-quant can. The trade-off is more complex dequantization, which is why they can decode a little slower on some hardware (Bandwidth vs compute explains why decode work matters).
What does an imatrix actually do?
An imatrix (importance matrix) is produced by running sample text through the model and recording which weights are activated most — i.e. which ones are "important". During quantization, that map is used to allocate precision more intelligently.
Key points:
- It helps most at low bit-rates, and I-quants in particular are usually built with an imatrix.
- A quant built with a good imatrix generally reads better than the same level built without one — but the calibration data matters, so it is not magic.
Which one should I actually download?
- 4-bit and up: a K-quant (
Q4_K_M/Q5_K_M/Q6_K) is the straightforward choice — see Which quant should I pick? for the full ladder. - 2–3 bit, memory-starved: reach for an I-quant, ideally an imatrix build, and accept a possible small decode-speed cost.
- Near-lossless: legacy
Q8_0remains a fine option when memory is no object.