You don’t need expensive hardware to run AI locally. Here are the cheapest setups that actually work, from $0 to $300.
$0: Your existing laptop
If you have a laptop with 8GB+ RAM made in the last 5 years, you can run AI right now.
curl -fsSL https://ollama.com/install.sh | sh
ollama run qwen3.5:4bWhat you get: A 4B model that handles basic coding questions, text summarization, and chat. Runs at 5-15 tok/s depending on your CPU.
What you don’t get: Fast responses, large models, or GPU-accelerated inference.
This is the “try before you buy” option. If you find yourself wanting more, then invest in hardware.
$80: Raspberry Pi 5 (8GB)
A dedicated AI device that runs 24/7 without noise or significant power draw.
- Runs Qwen3.5-0.8B at ~10 tok/s
- Add a $25 NVMe SSD for fast model loading
- Total: ~$105
Best for: Home assistant, IoT integration, learning, always-on AI server for simple tasks.
$150-200: Used mini PC (16GB RAM)
A used Dell OptiPlex, Lenovo ThinkCentre, or HP ProDesk with 16GB RAM and an i5/i7 processor. Available on eBay and Facebook Marketplace.
- Runs Qwen3.5-9B at ~8-12 tok/s (CPU)
- Silent, low power, small form factor
- Total: $150-200
Best for: Dedicated AI server, better than a Pi for daily use, runs 9B models that are genuinely useful.
$200-300: Used RTX 3060 12GB
The best bang-for-buck GPU for AI. Available used for $200-300. Put it in any desktop PC with a PCIe slot and a 500W+ power supply.
- Runs Qwen3.5-9B at ~20-30 tok/s (GPU)
- Runs DeepSeek Coder V2 Lite at ~25 tok/s
- 12GB VRAM handles models up to ~14B
- Total: $200-300 (assuming you have a desktop PC)
Best for: Serious local AI on a budget. The jump from CPU to GPU inference is transformative — 3-5x faster.
$300: Used RTX 3090 24GB
The secret weapon. The RTX 3090 has 24GB VRAM — same as the RTX 4090 — and can be found used for $500-700. But if you’re patient, deals pop up for $400-500.
- Runs Qwen 2.5 Coder 32B (best open-source coding model)
- Runs Qwen3.5-27B (strong all-rounder)
- 24GB VRAM handles models up to ~32B
- Total: $400-500
Best for: The best value in AI hardware. 24GB VRAM at half the price of a 4090.
Cost comparison vs API
| Setup | One-time cost | Monthly cost | Equivalent API spend |
|---|---|---|---|
| Existing laptop | $0 | $0 (electricity) | Pays for itself immediately |
| Used RTX 3060 | $250 | ~$5 electricity | Pays for itself in 2-3 months vs Claude |
| Used RTX 3090 | $500 | ~$8 electricity | Pays for itself in 3-4 months vs Claude |
| Mac Mini M4 32GB | $1,149 | ~$3 electricity | Pays for itself in 6-12 months |
If you’re spending $50+/month on AI APIs, any of these setups pays for itself within a year.
The recommendation
| Budget | Buy | Run |
|---|---|---|
| $0 | Nothing | Qwen3.5-4B on your laptop |
| $100 | Pi 5 + SSD | Qwen3.5-0.8B (dedicated device) |
| $200 | Used mini PC 16GB | Qwen3.5-9B (CPU) |
| $300 | Used RTX 3060 12GB | Qwen3.5-9B (GPU, 3-5x faster) |
| $500 | Used RTX 3090 24GB | Qwen 2.5 Coder 32B (best coding) |
The used RTX 3060 at $200-300 is the sweet spot. It transforms AI from “slow but works” to “fast and genuinely useful.”