The best value in AI hardware isn’t new — it’s used. A used RTX 3090 with 24GB VRAM costs $400-500 and runs the same models as a $1,600 RTX 4090. Here’s what to buy and what to avoid.
Best used GPUs for AI
RTX 3060 12GB — Best budget ($200-300)
The entry point for GPU-accelerated AI. 12GB VRAM runs models up to ~14B parameters.
- VRAM: 12GB GDDR6
- Models: Qwen3.5-9B, DeepSeek Coder V2 Lite, MiMo-V2-Flash (tight)
- Speed: ~20-30 tok/s on 9B models
- Power: 170W TDP
- Why buy: Cheapest way to get GPU inference. Massive upgrade over CPU-only.
RTX 3090 24GB — Best value ($400-600)
The sweet spot. 24GB VRAM matches the RTX 4090 on capacity and runs all the best open-source models.
- VRAM: 24GB GDDR6X
- Models: Qwen 2.5 Coder 32B, Qwen3.5-27B, Codestral, everything up to ~32B
- Speed: ~25-35 tok/s on 32B models
- Power: 350W TDP (needs a beefy PSU)
- Why buy: 24GB VRAM at half the price of a 4090. The best deal in AI hardware.
RTX 3080 Ti 12GB — Avoid for AI
Same 12GB as the 3060 but costs more used. The extra CUDA cores don’t help much for AI inference — VRAM is the bottleneck. Buy a 3060 instead and save $100+.
RTX 4090 24GB — Best performance ($1,200-1,600 used)
If you want the fastest consumer GPU, a used 4090 is ~$400 cheaper than new. Same 24GB VRAM as the 3090 but significantly faster inference.
- Speed: ~45 tok/s on 32B models (vs 3090’s ~25-35)
- Power: 450W TDP
- Why buy: If speed matters more than budget.
Tesla A100 40GB/80GB — Enterprise ($2,000-4,000 used)
If you need more than 24GB VRAM, used A100s from decommissioned data centers are the best option.
- 40GB version: runs 70B models, ~$2,000 used
- 80GB version: runs nearly anything, ~$3,500 used
- No display output (headless server card)
- Needs a server chassis or workstation with the right cooling
What to avoid
- Any GPU with less than 8GB VRAM. Too small for useful models.
- RTX 3080 10GB. Awkward VRAM size — too small for 14B models, too expensive vs 3060.
- Mining GPUs. Cards that were used for crypto mining 24/7 have reduced lifespan. Check for signs of heavy use.
- AMD GPUs. ROCm support is improving but still has compatibility issues with many AI tools. Stick to NVIDIA.
Where to buy
- eBay: Largest selection, buyer protection. Filter by “used” and sort by price.
- Facebook Marketplace: Often cheaper than eBay, but no buyer protection. Meet locally.
- r/hardwareswap: Reddit’s hardware trading community. Good deals, community reputation system.
- Refurbished from manufacturers: NVIDIA and partners sometimes sell refurbished cards with warranty.
What to check before buying
- VRAM size. This is the #1 spec that matters for AI. More VRAM = larger models.
- Seller reputation. Check feedback scores and history.
- Mining history. Ask if the card was used for mining. Miners often ran cards at high temperatures 24/7.
- Physical condition. Check for bent pins, damaged fans, thermal paste condition.
- Test it. Run a benchmark (GPU-Z, FurMark) immediately after receiving. Return if performance is below spec.
- Power supply. The 3090 needs 350W and two 8-pin connectors. Make sure your PSU can handle it.
The recommendation
| Budget | Buy | What it runs |
|---|---|---|
| $200-300 | Used RTX 3060 12GB | 9-14B models, great entry point |
| $400-600 | Used RTX 3090 24GB | 27-32B models, best value |
| $1,200-1,600 | Used RTX 4090 24GB | Same models as 3090, 40% faster |
| $2,000+ | Used A100 40GB | 70B models, enterprise grade |
For most people: used RTX 3090 for $400-500. It’s the best deal in AI hardware and will remain useful for years.
FAQ
Is a used GPU good for AI?
Yes — used GPUs offer the best value for local AI. A used RTX 3090 at $400–500 delivers the same 24GB VRAM and model compatibility as a new RTX 4090 at 3x the price, making it the sweet spot for most users.
Which used GPU is best for LLMs?
The RTX 3090 with 24GB VRAM is the best overall value for LLMs. It runs all popular open-source models up to ~32B parameters and costs $400–600 used — half the price of a new 4090 with the same VRAM capacity.
How much VRAM do I need?
12GB runs models up to ~14B parameters, 24GB handles up to ~32B models comfortably, and 40–80GB (A100) is needed for 70B+ models. VRAM is the single most important spec for AI — it determines the maximum model size you can run. If you need more VRAM than you want to buy, cloud GPU providers offer 40–80GB instances on demand.
Should I buy used or new?
Buy used unless you need warranty or the absolute fastest inference speed. A used RTX 3090 at $400–500 matches a new RTX 4090’s VRAM capacity at one-third the cost. Only buy new if you specifically need the 4090’s faster inference or can’t find a reputable used seller.
Related
- Best GPU for Running AI Models Locally in 2026
- Cheapest Way to Run AI Locally in 2026
- How Much VRAM Do You Need for AI?
- Best Self-Hosted AI Models in 2026
Related: Best AI Engineering Courses