Ollama vs llama.cpp vs vLLM vs LM Studio — which and when
Four popular ways to run models locally, and which one fits single-user desktop use versus multi-user serving, by GPU vendor and OS. Opinionated but fair.
Four names come up again and again. They overlap, but each has a sweet spot. Here is an opinionated-but-fair guide to choosing.
llama.cpp — the flexible engine
The engine much of the ecosystem is built on. It runs GGUF models on CPU, GPU, or a split of both, across every vendor path — CUDA, ROCm, Vulkan, Metal and SYCL (GPU software stacks). It ships a CLI and a server. Reach for it when you want control, need to offload layers to CPU because a model barely fits, or you are on non-NVIDIA hardware.
Ollama — the easy on-ramp
A friendly wrapper built on llama.cpp. One command pulls a model and starts a simple local API with sensible defaults. Ideal for a single user getting started or wiring a model into a small app, at the cost of less low-level control.
LM Studio — the desktop GUI
A graphical desktop app (also llama.cpp-based, with MLX on Apple Silicon). It gives you a model browser, a chat UI, and a local OpenAI-compatible server. Great for single-user, non-technical use on Windows, macOS or Linux, and it uses Metal on Macs.
vLLM — the serving engine
Built for production, multi-user serving. It is GPU-first (CUDA, with some ROCm), does high-throughput batched inference (PagedAttention), and exposes an OpenAI-compatible API. It centres on GPU formats like AWQ/GPTQ/FP8 rather than GGUF. Overkill for one person; the right tool when you are serving many concurrent requests.
Choosing
- Single user, easiest: Ollama or LM Studio.
- Single user, want control / offload / non-NVIDIA: llama.cpp.
- On a Mac: any of llama.cpp, Ollama or LM Studio — all use Metal; LM Studio adds MLX (which Mac chip).
- Serving an app / many users / max throughput on NVIDIA: vLLM.
Which model format you download depends on the runtime — GGUF for the llama.cpp trio, GPU formats for vLLM. See Quantization formats explained.