Mixtral-8x7B-Instruct-v0.1 — performance reports

Community-submitted & scraped throughput measurements. Sortable by generation speed.

Ran this model? Log in to submit your own performance result.

Filter by GPU / rig

Sort by

Rig / GPU	Quant	Runtime	Context	Prompt tok/s	Gen tok/s	VRAM (GB)	Source	Reporter
RTX 4090	AWQ · Q4_K_M	llama.cpp	32,768	545.96	18.52	75.72	user	Mr. Zachariah Bahringer MD