Whether a given AMD or Intel GPU is a good local-LLM card is a moving target: the software stack (ROCm, Vulkan, SYCL / IPEX-LLM) changes month to month, and the fastest backend often flips by architecture. This board tracks the current picture per backend, with dated, sourced entries — Apple Metal is included only for contrast.
The most complete AMD path, and the one to target on Linux. But the fastest backend now depends on your architecture: RDNA3 favours ROCm/HIP, while on RDNA4 a Vulkan backend has measured faster token generation. Consumer-card coverage keeps widening release to release.
FlashAttention-3 remains effectively NVIDIA-Hopper-exclusive. AMD has FA-2-equivalents (AOTriton / CK / Triton), best on CDNA MI300X; RDNA3 is experimental and consumer RDNA is weak — enabling FA on RDNA3 without rocWMMA can actually reduce llama.cpp speed. NVLink, TensorRT-LLM and ExLlamaV2 stay NVIDIA-only.
On RDNA3 (Linux), ROCm/HIP beats Vulkan: an RX 7800 XT ran prompt processing with FlashAttention at 2304 vs 2064 t/s, and an RX 7600 XT was about 80% faster at pp512 under ROCm. On RDNA3, prefer ROCm — the opposite of the RDNA4 picture.
ROCm 7.14 became AMD's production release, built through the modular "TheRock" system. It supersedes the 7.2.x line and adds Ryzen AI 400-series support. (The 7.2 to 7.14 version jump is real, not a typo.)
ROCm 7.2.0 expanded consumer Radeon coverage on Linux: RX 9070 / 9070 XT (RDNA4), RX 7900 XTX / XT (RDNA3), plus RX 7700, R9600D and 9060 XT LP.
Runs on recent Radeon, but support is tiered by architecture: full support (including the debugger) only on RDNA4, runtime + SDK without a debugger on RDNA3, and RDNA2 unsupported. On Windows, a Vulkan backend is often faster than HIP — the old "Vulkan is ~80-90% of ROCm" rule no longer holds.
Correction to earlier guidance: the old "Vulkan reaches ~80-90% of ROCm on Windows" rule of thumb no longer holds — on RDNA4 that relationship has inverted, with Vulkan often ahead. Do not rely on a single fixed ratio.
On Windows RDNA4, community testing reports Vulkan beating ROCm/HIP by roughly 14-30% (Wave32 vs Wave64), reaching about 90 tok/s on an 8B Q4_K_M model.
The Windows HIP SDK 7.1.1 (production ROCm 7.2.4) gives full support including the debugger only on RDNA4 (RX 9070 XT / 9070 / 9070 GRE, gfx1201; 9060 XT / 9060, gfx1200). RDNA3 gets runtime + SDK but no debugger; RDNA2 (RX 6000) is unsupported.
A vendor-neutral backend that runs well across AMD and Intel and is frequently the fastest option on RDNA4 and on Windows. It is not a universal winner: ROCm/HIP still leads on RDNA3, and Intel SYCL can crush it on prompt processing. There is no single Vulkan-vs-native ratio.
The Vulkan-vs-ROCm relationship inverts by architecture: Vulkan tends to lead on RDNA4, while ROCm/HIP leads on RDNA3. Treat any single "Vulkan is X% of ROCm" number with suspicion — it depends on the card.
On RDNA4 (Linux), the RADV Vulkan backend measured about 18.6% faster token generation than ROCm/HIP on qwen3:8b (100.0 vs 84.3 tok/s) and about 9.5% faster on qwen2.5-coder:14b; the root cause was a gfx-target mismatch in the ROCm build.
LM Studio 0.4.17 (June 2026) shipped llama.cpp runtime v2.22.1, adding Strix Halo and R9600D / R9700 support and fixing AMD GPUs going undetected after a driver update.
Intel Arc runs today's models through IPEX-LLM (SYCL) or Vulkan, and IPEX-LLM is actively maintained. Which backend is faster depends on the metric and build: SYCL can dominate prompt processing while Vulkan can lead decode. Resizable BAR is effectively mandatory.
IPEX-LLM is actively maintained, carrying llama.cpp, Ollama and vLLM on the Intel XPU stack.
Intel Arc needs Resizable BAR enabled: without it, community testing reports roughly a 20-25% performance penalty.
Correction to earlier guidance: the old "Intel Vulkan beats SYCL by ~2x" claim does not hold generally — it varies by metric and build (SYCL can dominate prompt processing while Vulkan can lead decode).
But other community tests put Vulkan ahead on decode for the Arc B580 (~40-42 vs 32-38 tok/s on Linux native; ~15-20 on the Windows Portable ZIP). The fastest Intel path varies by metric and build — there is no single ratio.
On the Intel Arc B580, native IPEX-LLM (SYCL) crushes Vulkan on prompt processing: pp512 of 899.7 vs 119.2 t/s (about 7.5x), and tg128 of 26.9 vs 21.5 tok/s.
Included for contrast: Apple Silicon is a smooth, well-supported path. MLX has overtaken llama.cpp's Metal backend on decode (Ollama switched its Apple backend to MLX), while raw Metal still wins prompt-heavy work.
Apple's M5 adds per-GPU-core Neural Accelerators, reported at roughly 4x the time-to-first-token of the M4 for MLX workloads.
MLX overtook llama.cpp's Metal backend on decode, and Ollama switched its Apple backend to MLX around March 2026. On an M4 Max, Qwen3.5-35B ran about 130 tok/s under MLX vs 89.4 under raw llama.cpp Metal — though Metal still wins prefill / prompt-heavy work.
selected to compare · pick at least 2