Qwen3 4B
4B · chat, reasoning · apache-2.0
- Weights
- 4.3 GB
- KV @ 8K
- 1.2 GB
- Total
- 6.0 GB
llama-server -hf unsloth/Qwen3-4B-GGUF:Q8_0 -c 8192Local AI
Which LLM runs on my machine?
Pick your hardware. For each open model we work out the biggest quant that fits, using real GGUF file sizes from Hugging Face plus the KV cache for your context length.
8 of 9 models fit in about 11.2 GB usable. macOS lets the GPU use roughly 70% of unified memory by default.
Q8_0
Phi-4 Mini Instruct
3.8B · chat, reasoning · mit
- Weights
- 3.2 GB
- KV @ 8K
- 1.1 GB
- Total
- 4.7 GB
llama-server -hf unsloth/Phi-4-mini-instruct-GGUF:Q6_K -c 8192Qwen3 8B
8B · chat, reasoning · apache-2.0
- Weights
- 8.7 GB
- KV @ 8K
- 1.2 GB
- Total
- 10.4 GB
llama-server -hf unsloth/Qwen3-8B-GGUF:Q8_0 -c 8192Qwen2.5 Coder 7B Instruct
7B · coding · apache-2.0
- Weights
- 8.1 GB
- KV @ 8K
- 0.5 GB
- Total
- 9.1 GB
llama-server -hf unsloth/Qwen2.5-Coder-7B-Instruct-GGUF:Q8_0 -c 8192DeepSeek-R1 Distill Qwen 7B
7B · reasoning · apache-2.0
- Weights
- 8.1 GB
- KV @ 8K
- 0.5 GB
- Total
- 9.1 GB
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF:Q8_0 -c 8192Qwen3 14B
14B · chat, reasoning · apache-2.0
- Weights
- 9.0 GB
- KV @ 8K
- 1.3 GB
- Total
- 10.8 GB
llama-server -hf unsloth/Qwen3-14B-GGUF:Q4_K_M -c 8192Qwen2.5 Coder 14B Instruct
14B · coding · apache-2.0
- Weights
- 9.0 GB
- KV @ 8K
- 1.6 GB
- Total
- 11.1 GB
llama-server -hf unsloth/Qwen2.5-Coder-14B-Instruct-GGUF:Q4_K_M -c 8192DeepSeek-R1 Distill Qwen 14B
14B · reasoning · apache-2.0
- Weights
- 9.0 GB
- KV @ 8K
- 1.6 GB
- Total
- 11.1 GB
llama-server -hf unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF:Q4_K_M -c 8192Qwen3 30B-A3B (MoE)
30B (3B active) · chat, reasoning, coding · apache-2.0
Needs at least 16.0 GB at 8K context.
All quants on Hugging Face →Estimates only: weights + fp16 KV cache + 0.5 GB runtime buffers. Real usage varies with the runtime, batch size and KV-cache quantization. Model data refreshed 2026-10-18.