The NVIDIA Jetson Orin Nano is the most capable edge AI computer you can buy for under $250. At 67 TOPS of AI performance in a credit-card-sized board, it runs quantized LLMs, vision models, and speech recognition locally, without cloud APIs.
If you’re building AI that needs to run on-device (robots, drones, IoT, kiosks, vehicles), the Jetson Orin Nano is the default choice. Here’s why.
What is the Jetson Orin Nano?
The Jetson Orin Nano is NVIDIA’s entry-level edge AI module. It’s a complete computer on a small board: CPU, GPU, memory, storage, and I/O, all designed for AI inference at the edge.
The “Super” variant (released late 2024) bumped performance significantly: 67 TOPS (up from 40 TOPS) while keeping the same $249 price point. For developers used to running AI on cloud APIs, this changes the economics of on-device inference.
Specs
| Spec | Jetson Orin Nano 4GB | Jetson Orin Nano 8GB | Jetson Orin Nano Super |
|---|---|---|---|
| AI Performance | 20 TOPS | 40 TOPS | 67 TOPS |
| GPU | 1024-core NVIDIA Ampere | 1024-core NVIDIA Ampere | 1024-core NVIDIA Ampere |
| GPU Clock | 625 MHz | 625 MHz | 1020 MHz |
| CPU | 6-core Arm Cortex-A78AE | 6-core Arm Cortex-A78AE | 6-core Arm Cortex-A78AE |
| Memory | 4GB LPDDR5 | 8GB LPDDR5 | 8GB LPDDR5 |
| Storage | microSD / NVMe | microSD / NVMe | microSD / NVMe |
| Power | 7W / 15W | 7W / 15W | 7W / 25W |
| Price | $199 | $249 | $249 |
The Super variant at $249 is the sweet spot. 67 TOPS and 8GB LPDDR5 for the same price as the original 8GB model.
What can you run on it?
LLMs
With 8GB LPDDR5 and GPU offloading, you can run:
| Model | Params | Quantization | Speed | Usable? |
|---|---|---|---|---|
| Llama 3.2 3B | 3B | Q4_K_M | 8-12 tok/s | Yes |
| Gemma 4 2B | 2B | Q4_K_M | 10-15 tok/s | Yes |
| Qwen 3.6 1.5B | 1.5B | Q4_K_M | 15-20 tok/s | Yes |
| Phi-3 Mini | 3.8B | Q4_K_M | 6-10 tok/s | Yes |
| Llama 3.1 8B | 8B | Q2_K | 2-4 tok/s | Barely |
The 8GB memory limit is the bottleneck. You cannot run 7B+ models at reasonable quality. But 1.5B-3B models run well, and for specific tasks (classification, extraction, simple Q&A), they’re sufficient.
Vision models
The GPU handles vision tasks well:
- YOLO v8: Real-time object detection at 30+ FPS
- CLIP: Image-text matching, search
- Stable Diffusion: 512x512 images in ~30 seconds
- Whisper: Real-time speech recognition
TensorRT optimization
NVIDIA’s TensorRT framework optimizes models specifically for Jetson hardware. Converting a model to TensorRT typically gives 2-5x speedup:
# Install TensorRT
sudo apt install tensorrt
# Convert ONNX model to TensorRT
trtexec --onnx=model.onnx --saveEngine=model.trt --fp16Setup
Initial setup
- Flash the Jetson Orin Nano with JetPack SDK
- Insert microSD or NVMe SSD
- Connect power, monitor, keyboard
- Run the initial setup wizard
# Update system
sudo apt update && sudo apt upgrade -y
# Install JetPack components
sudo apt install nvidia-jetpack -y
# Verify GPU
jtopInstall Ollama
# Install Ollama for ARM64
curl -fsSL https://ollama.com/install.sh | sh
# Pull a small model
ollama pull gemma4:2b
# Test
ollama run gemma4:2b "Hello, world!"Install Python AI stack
# Install pip packages
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
pip install transformers accelerate
# Test inference
python3 -c "
from transformers import pipeline
pipe = pipeline('text-generation', model='Qwen/Qwen3.6-0.5B')
print(pipe('Hello, world!', max_new_tokens=50))
"Performance benchmarks
| Task | Model | Speed | Notes |
|---|---|---|---|
| Text generation | Gemma 4 2B Q4 | 12 tok/s | Good for chat |
| Text generation | Llama 3.2 3B Q4 | 9 tok/s | Good for coding |
| Object detection | YOLO v8 Nano | 35 FPS | Real-time |
| Speech recognition | Whisper Base | 2x realtime | Batch processing |
| Image generation | SD 1.5 | ~30s/image | 512x512 |
Power consumption
| Mode | Power Draw | Use Case |
|---|---|---|
| 7W | Idle/light inference | Battery-powered, always-on |
| 15W | Standard inference | Mains-powered, balanced |
| 25W (Super) | Max performance | When you need full speed |
At 15W, the Jetson Orin Nano costs about $0.50/month to run 24/7 (at $0.15/kWh). Compare that to cloud API costs.
When to use the Jetson Orin Nano
- You need on-device AI inference (no cloud dependency)
- You’re building robots, drones, IoT devices, or kiosks
- You need real-time vision processing
- Power consumption matters (7-25W)
- You need NVIDIA’s CUDA/TensorRT ecosystem
When to use something else
- More memory: Jetson Orin NX (16GB) or Jetson AGX Orin (64GB)
- Cheaper: Raspberry Pi 5 + AI HAT ($100-150 total)
- Desktop inference: Any GPU-equipped PC with Ollama
- Cloud inference: GPT-5.6 Luna at $0.20/$1.20 per 1M tokens
My take
The Jetson Orin Nano Super at $249 is the best edge AI computer for most developers. 67 TOPS and 8GB LPDDR5 handles quantized LLMs, vision models, and speech recognition at reasonable speeds.
The limitation is memory. 8GB means you cannot run 7B+ models at good quality. For larger models, you need the Jetson Orin NX (16GB, $499) or Jetson AGX Orin (64GB, $999+).
For developers building AI that needs to run on-device, the Jetson Orin Nano is the default choice. The NVIDIA software ecosystem (CUDA, TensorRT, JetPack) is mature and well-documented. No other edge AI platform comes close.
FAQ
Can the Jetson Orin Nano run LLMs?
Yes, but limited to 1.5B-3B models at reasonable quality. The 8GB LPDDR5 memory is the bottleneck. Larger models require the Orin NX (16GB) or AGX Orin (64GB).
How fast is the Jetson Orin Nano?
67 TOPS of AI performance (Super variant). For LLMs, expect 8-15 tok/s with 2B-3B models. For vision, 30+ FPS with YOLO v8.
Can I use the Jetson Orin Nano as a desktop computer?
Technically yes, but it’s not designed for that. It runs Ubuntu Linux and has HDMI output, but the Arm CPU and 8GB memory make it slow for general desktop use. It’s an AI inference device, not a PC replacement.
How much does it cost to run 24/7?
At 15W, about $0.50/month (at $0.15/kWh). At 25W (Super max), about $0.90/month. Much cheaper than cloud API costs for equivalent workloads.
Can I run Ollama on the Jetson Orin Nano?
Yes. Ollama supports ARM64 and runs well on the Jetson. Pull small models (1.5B-3B) for best performance.
What’s the difference between Jetson Orin Nano and Super?
The Super variant has a higher GPU clock (1020 MHz vs 625 MHz), delivering 67 TOPS vs 40 TOPS. Same price ($249), same memory (8GB), higher power draw (25W max vs 15W). The Super is the better buy.