Project ideas by hardware tier: what to actually run locally
You bought the hardware — now what? Realistic local-LLM project ideas grouped by rig, from an 8 GB CPU-only laptop to a multi-GPU box, each with an example catalog model, a runtime, and honest expectations.
8 min read Reviewed July 2026
On this page
TL;DR
Bought the hardware, now what? This hub groups realistic local-LLM projects by the rig you have — from an 8 GB CPU-only laptop to a multi-GPU box. For each tier you get project ideas, an example model from our catalog to run each with, the runtime to use, and an honest note on what to expect. Rule of one: pick your tier, pick a project, then check the exact model in the calculator before you download — fit depends on your quant and context, not just the model name.
How do I know which tier I'm in?
Your tier is set by the memory that holds the model — dedicated GPU VRAM on a PC, the unified memory shared with the GPU on an Apple Silicon Mac, or plain system RAM if you have no usable GPU.
Find your tier, top down
- No dedicated GPU? You're CPU-only — go by system RAM (the 8 GB tier below).
- Apple Silicon Mac? Use the unified-memory tier — your whole RAM pool is usable.
- PC with one GPU? Find your card's VRAM (8, 12-16, or 24 GB) — that's your tier.
- Two or more GPUs? You're in the multi-GPU tier — pooled VRAM, with the caveats in that section.
The number that matters is how much memory the weights and the context/KV cache need together. A smaller quant shrinks the weights; a longer context grows the cache.
8 GB RAM, CPU-only laptop
No graphics card, modest RAM. You can still run small models entirely on the CPU with Ollama or llama.cpp — just at CPU speed, so lean on small models and, where you can, ones with few active parameters.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| Private offline chat / drafting help | Phi-4-mini-instruct (or lighter, Qwen3-1.7B), Ollama | Works, but responses arrive slower than reading speed; keep context short |
| Summarize notes on a budget | granite-4.0-h-tiny, a mixture-of-experts model with only ~1.5B active, llama.cpp | Few active params make it friendlier on CPU; good for a page, slow on long docs |
| Scan-to-text (document OCR) | dots.ocr, run as a batch | Fine for processing a stack of pages in the background, not for instant results |
8 GB VRAM (entry discrete GPU)
A budget or older gaming card. Small models now run fully on the GPU and feel responsive; leave headroom for context.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| Snappy private chat | Llama-3.1-8B-Instruct at a small quant, Ollama or LM Studio | Fast and comfortable — an 8B model on a GPU is the sweet spot here |
| RAG over your personal notes | Qwen3-4B plus a small embedder like Qwen3-Embedding-0.6B, Ollama | Solid retrieval-augmented answers over a modest corpus; watch context length |
| Image and voice tinkering | gemma-4-E2B-it, a small multimodal model, LM Studio | Handles text, images and audio on entry hardware; keep inputs and context modest |
12-16 GB VRAM (mainstream GPU)
The comfortable mainstream tier. Capable 14B-class models run well, and small-active MoE models punch above their footprint.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| Capable everyday assistant | Qwen3-14B, Ollama | A real step up in quality with room for a real conversation; very long context still eats VRAM |
| Fast MoE assistant | gpt-oss-20b, an MoE with only ~4.8B active, LM Studio | Fast for its size; all experts must still fit in VRAM, so check the calculator |
| Screenshot / image Q&A | Kimi-VL-A3B-Instruct, a ~16B vision MoE with a small active set | Vision plus text; the whole ~16B of experts must fit, so it wants the upper end of this tier |
24 GB VRAM (one large card)
A single high-VRAM card. This is where capable 24-32B models run comfortably and agentic or coding workflows get genuinely useful.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| Coding assistant in your editor | Devstral-Small-2-24B, Ollama or LM Studio | A real local coding helper; good for edits and questions, not a hosted-frontier replacement |
| Agentic tool-use coding | Qwen3-Coder-30B-A3B, an MoE with only ~3.4B active, llama.cpp | Fast for its size thanks to few active params; keep tool loops and context bounded |
| Document and image understanding | Qwen3-VL-32B-Instruct, check fit at your quant | Strong vision plus text; 32B is a tight fit at 24 GB, so mind quant and context |
Mac with unified memory (Apple Silicon)
Apple Silicon shares one big memory pool between CPU and GPU, so a Mac with lots of RAM can hold models a similarly priced PC GPU cannot — especially mixture-of-experts models, which need memory for all experts but only activate a few per token.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| Big-MoE personal assistant | gpt-oss-120b, an MoE with only ~9.3B active, llama.cpp / LM Studio | A large model that fits because few params are active per token; needs plenty of unified RAM |
| Long-document summarizing pipeline | Qwen3-30B-A3B-Instruct, run as a batch | Great for chewing through documents overnight; still not instant |
| Large-context agentic workflow | Llama-4-Scout-17B-16E, a 108B-total MoE with about 18B active | Big-model quality with modest active compute; how fast it feels tracks your chip's memory bandwidth |
Multi-GPU / 48 GB+ (enthusiast and workstation)
Two or more cards, or a workstation card, pooling VRAM in a multi-GPU setup. This unlocks 70B-class dense models and serious agentic work — with the caveat that a second card adds memory, not proportional speed.
| Project idea | What it needs | Realistic expectation |
|---|---|---|
| 70B-class private assistant | Llama-3.3-70B-Instruct, split across cards | Excellent quality locally; a layer split decodes at roughly one card's speed, not double |
| Serious coding co-pilot | Kimi-Dev-72B, llama.cpp or vLLM | Strong 72B coding model; needs the pooled VRAM and patience for long generations |
| Heavier agentic pipelines | Hunyuan-A13B, an 80B-total MoE | Pooled VRAM holds the whole MoE while decode stays near the small active-param speed |
Which tier do I actually need?
Start from the project, not the hardware. A private chat assistant or note summarizer is happy on modest hardware; 70B-class assistants and heavy coding models need 24 GB, a big-RAM Mac, or multiple GPUs. Pick a project above, then confirm the specific model in the calculator.
Is a small model good enough for real work?
Often, yes. Modern 2-8B models like Qwen3-4B, Phi-4-mini-instruct and gemma-4-E2B-it handle chat, drafting and note-summarizing well, and the catalog carries dedicated embedding and reranker models for RAG. Reach for a bigger model when you hit a real limit, not by default.
Will a bigger model always be better?
No. Once a model only fits at a very low quant, a smaller model at a healthy quant often produces better everyday output. Fit is about weights and context together — see which quant to pick.
Do these ideas need an internet connection?
No — that is the point of local. Once you have downloaded the model, chat, RAG, OCR and coding help all run offline on your own machine.
Sources & data
- Model parameter counts and active-parameter (MoE) figures: the LLM Torrents model catalog (paged in full, 2026-07). Whether a given model fits a given rig is computed live by the calculator, not asserted here.