Jul 29, 2026 · 9 min read

How to Run FLUX Locally: Generate AI Images for Free on Your GPU (2026)

FLUX is the best open-weight image generation model family in 2026. Running it locally means unlimited image generation at zero marginal cost. No API fees, no rate limits, no data leaving your machine. I generate 50 to 100 images per day for blog covers, social media, and design mockups without spending a cent.

This guide covers every FLUX variant, the exact hardware you need, how to set it up with ComfyUI, and real speed benchmarks from my own hardware. If you have a GPU with 8GB or more VRAM (or an Apple Silicon Mac with 16GB+), you can run FLUX today.

FLUX Model Family Overview

The FLUX family from Black Forest Labs has three main variants you can run locally:

Model	Parameters	Min VRAM	Quality	Speed	License
FLUX.1 [dev]	12B	8GB (Q4)	8.5/10	Medium	Non-commercial
FLUX.2 [dev]	32B	19GB (Q4 GGUF)	9.5/10	Slow	Non-commercial
FLUX.2 [klein] 4B	4B	8GB	8/10	Fast	Apache 2.0

My recommendation:

8GB VRAM: Start with FLUX.2 [klein] 4B. It generates images in 4 steps and the quality is shockingly good for a 4B model.
12GB VRAM: Run FLUX.1 [dev] at Q4 quantization. Better quality than klein, reasonable speed.
24GB VRAM: Run FLUX.2 [dev] at FP8. The best quality you can get locally.
Apple Silicon 16GB+: FLUX.1 [dev] runs well through MLX or ComfyUI with unified memory.

VRAM Requirements by GPU

Here is exactly what you can run on common hardware:

GPU	VRAM	Best FLUX Model	Quantization	Notes
RTX 3060	12GB	FLUX.1 [dev]	Q4_K_M	Good speed, good quality
RTX 4060	8GB	FLUX.2 [klein] 4B	FP16	Full precision, fast
RTX 4060 Ti	16GB	FLUX.1 [dev]	Q8	Great balance
RTX 4070	12GB	FLUX.1 [dev]	Q4_K_M	Fast inference
RTX 4070 Ti	16GB	FLUX.2 [dev]	Q4 GGUF	Near-best quality
RTX 4090	24GB	FLUX.2 [dev]	FP8	Best possible quality
M2 Mac 16GB	16GB shared	FLUX.1 [dev]	Q4	Works via MLX
M2 Pro 32GB	32GB shared	FLUX.2 [dev]	Q4 GGUF	Full model fits
M3 Max 64GB	64GB shared	FLUX.2 [dev]	FP16	Maximum quality

For a deeper understanding of VRAM requirements across all AI workloads, check the how much VRAM for AI guide.

Setup with ComfyUI (Recommended)

ComfyUI is the best interface for running FLUX locally. It is node-based, highly configurable, and has excellent community workflows for every FLUX variant.

Step 1: Install ComfyUI

# Clone the repository
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On macOS/Linux
# venv\Scripts\activate  # On Windows

# Install dependencies
pip install -r requirements.txt

# For NVIDIA GPUs, install PyTorch with CUDA
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Step 2: Download FLUX Models

For FLUX.2 [klein] 4B (easiest start, 8GB VRAM):

# Download to ComfyUI models folder
cd models/checkpoints/
wget https://huggingface.co/black-forest-labs/FLUX.2-klein/resolve/main/flux2-klein-4b.safetensors

For FLUX.1 [dev] (Q4 quantized for 8 to 12GB VRAM):

cd models/checkpoints/
wget https://huggingface.co/city96/FLUX.1-dev-gguf/resolve/main/flux1-dev-Q4_K_M.gguf

For FLUX.2 [dev] (FP8 for 24GB VRAM):

cd models/checkpoints/
wget https://huggingface.co/black-forest-labs/FLUX.2-dev/resolve/main/flux2-dev-fp8.safetensors

Step 3: Download the Text Encoder

FLUX uses the T5-XXL text encoder (separate from the main model):

cd models/text_encoders/
wget https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/t5xxl_fp8_e4m3fn.safetensors
wget https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors

Step 4: Download the VAE

cd models/vae/
wget https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/ae.safetensors

Step 5: Launch ComfyUI

python main.py
# Opens at http://127.0.0.1:8188

Load the default FLUX workflow (included in ComfyUI’s examples folder) and you are generating images.

Setup with AUTOMATIC1111 (Alternative)

If you prefer the AUTOMATIC1111 WebUI (more familiar interface, less flexible):

# Clone and setup
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui

# Install (auto-downloads dependencies)
./webui.sh  # Linux/Mac
# webui-user.bat  # Windows

AUTOMATIC1111 supports FLUX through the Forge extension. However, I recommend ComfyUI for FLUX specifically because:

Better memory management (critical for large models)
More FLUX-specific workflows available
Quantized model support is more mature
LoRA loading is more reliable

Speed Benchmarks: Images Per Minute

I tested each setup with a standard prompt (“a photorealistic landscape with mountains and a lake at sunset, 4K quality”) at 1024x1024:

FLUX.2 [klein] 4B (4 steps):

Hardware	Images/Minute	Time Per Image
RTX 4060 8GB	12	5s
RTX 4070 12GB	18	3.3s
RTX 4090 24GB	30	2s
M2 Pro 32GB	6	10s

FLUX.1 [dev] Q4 (28 steps):

Hardware	Images/Minute	Time Per Image
RTX 3060 12GB	2.5	24s
RTX 4070 12GB	4	15s
RTX 4090 24GB	8	7.5s
M2 Pro 32GB	1.5	40s

FLUX.2 [dev] FP8 (28 steps):

Hardware	Images/Minute	Time Per Image
RTX 4090 24GB	3	20s
M3 Max 64GB	1.2	50s

The speed differences are dramatic. FLUX.2 [klein] at 4 steps is genuinely fast enough for interactive use. FLUX.2 [dev] is a “start generation and go get coffee” workflow.

Cost Comparison: Local vs API

Let me calculate the real cost of generating 1000 images per month:

Local (RTX 4090, already owned):

Electricity: ~$3 to $5/month (assuming $0.15/kWh, 350W for 20 hours)
API cost: $0
Total: $3 to $5 per 1000 images ($0.003 to $0.005 per image)

Local (rented GPU server, $0.50/hour):

RTX 4090 at 8 images/minute for FLUX.1 [dev]
1000 images = 125 minutes = ~$1.05
Total: ~$1 per 1000 images ($0.001 per image)

fal.ai API:

FLUX.2 [dev]: $0.03 per image
1000 images: $30
Total: $30 per 1000 images

OpenAI GPT-Image-1:

$0.04 per image
1000 images: $40
Total: $40 per 1000 images

Midjourney API:

~$0.08 per image
1000 images: $80
Total: $80 per 1000 images

The economics are clear: if you generate more than 200 to 300 images per month, self-hosting pays for itself versus APIs. The breakeven point depends on your existing hardware.

LoRA Training for Custom Styles

LoRAs (Low-Rank Adaptations) let you fine-tune FLUX for specific styles, subjects, or brands without retraining the full model.

Requirements:

12GB+ VRAM (minimum)
24GB recommended for comfortable training
10 to 50 training images
kohya_ss or ai-toolkit for training

Basic LoRA training setup:

# Install ai-toolkit
git clone https://github.com/ostris/ai-toolkit.git
cd ai-toolkit
pip install -r requirements.txt

# Prepare training images (place in a folder)
# 10-50 images of your target style/subject
# Resolution: 1024x1024 or higher

Training configuration (config.yaml):

model:
  name: flux-dev
  path: /path/to/flux1-dev.safetensors

training:
  resolution: 1024
  batch_size: 1
  epochs: 100
  learning_rate: 1e-4
  lora_rank: 16

dataset:
  path: /path/to/training/images
  caption_extension: .txt

Training a LoRA takes 1 to 3 hours on an RTX 4090, longer on smaller GPUs. The resulting LoRA file is typically 50 to 200MB and can be loaded into ComfyUI alongside the base model.

Use cases for LoRA training:

Consistent brand style across all generated images
Specific character or product appearance
Architectural visualization style matching
Custom illustration styles for publications

Optimizing for Your Hardware

If you have exactly 8GB VRAM:

Use FLUX.2 [klein] 4B. It is specifically designed for constrained hardware and uses only 4 inference steps (versus 28 for other FLUX models). The quality is remarkable for a 4B model.

If you have 12GB VRAM:

FLUX.1 [dev] at Q4_K_M quantization. Enable --lowvram mode in ComfyUI if you run out of memory. This offloads parts of the model to system RAM when not in use.

If you have 16GB VRAM:

FLUX.1 [dev] at Q8 quantization gives better quality than Q4 with the same model. Or run FLUX.2 [dev] at aggressive Q4 quantization if you want the newer model.

If you have 24GB VRAM:

FLUX.2 [dev] at FP8. This is the sweet spot for quality. No quantization artifacts, full model capability.

Apple Silicon tips:

Macs use unified memory, which means the “VRAM” is shared with system RAM. A 16GB Mac can run FLUX.1 [dev] at Q4, but you need to close other applications to free memory. A 32GB Mac runs it comfortably alongside your IDE and browser.

For Apple Silicon, consider using the MLX framework instead of PyTorch for 20 to 40% faster inference. ComfyUI supports MLX through community nodes.

Practical Workflow: Blog Cover Images

Here is my actual workflow for generating cover images for this blog:

Write a detailed prompt describing the image I want
Generate 4 variations with FLUX.1 [dev] locally (takes about 2 minutes on RTX 4070)
Pick the best one and upscale 2x with Real-ESRGAN (locally, free)
Minor adjustments in GIMP/Photoshop if needed
Export and compress for web

Total time: 5 to 10 minutes per cover image. Total cost: $0.

Compare this to using an API at $0.03/image: the cost would be $0.12 for 4 variations. Not much, but it adds up if you generate images frequently, and you lose the ability to iterate quickly without watching costs.

Integration with Your AI Dev Stack

If you are already running Ollama for coding AI, adding local image generation is natural. Both use GPU resources, so plan your VRAM budget:

12GB GPU: Run Ollama OR ComfyUI, not both simultaneously
24GB GPU: Run both, allocate 12GB to each
Apple Silicon 32GB+: Both run comfortably in shared memory

For developers using the best AI coding tools alongside image generation, the key is not running inference on both simultaneously unless you have the VRAM headroom. Tools like LM Studio can help manage model loading and unloading to share GPU resources.

This setup also complements the $0 AI coding stack or the privacy-first developer stack by adding image generation without any cloud dependency.

Troubleshooting Common Issues

Out of memory errors:

Reduce batch size to 1
Use a more aggressive quantization (Q4 instead of Q8)
Enable model offloading in ComfyUI settings
Close other GPU-using applications

Black or corrupted images:

Usually a quantization issue. Try a different quant level.
Ensure VAE is loaded correctly
Check that text encoder files are complete (not truncated downloads)

Slow generation on Mac:

Ensure you are using MPS (Metal) backend, not CPU
Install the latest PyTorch nightly for best Apple Silicon support
Consider MLX-based nodes for ComfyUI

FAQ

Which FLUX model should I start with?

If you have 8GB VRAM, start with FLUX.2 [klein] 4B. If you have 12GB+, start with FLUX.1 [dev] at Q4. The klein model generates images in 4 steps (fast) while FLUX.1 [dev] takes 28 steps (slower but higher quality). Both produce results that beat SDXL convincingly. Quality-wise, open source models have gotten remarkably good across both text and image generation.

Can I use FLUX commercially?

FLUX.2 [klein] is Apache 2.0 licensed and fully commercial use is permitted. FLUX.1 [dev] and FLUX.2 [dev] are non-commercial licenses (research and personal use only). For commercial projects, use [klein] locally or pay for FLUX [pro] through an API like fal.ai.

How does local FLUX compare to Midjourney?

For photorealistic images, FLUX.2 [dev] is within 5 to 10% of Midjourney v7 quality. Midjourney still has an edge on artistic and stylized outputs. For most developer use cases (product shots, blog images, UI mockups), FLUX locally is more than adequate and infinitely cheaper.

Do I need to train a LoRA for consistent style?

For blog covers and general use, no. Good prompting gets you 80% of the way to a consistent style. LoRA training is worth it when you need pixel-level consistency (same character across images, exact brand style matching, specific product appearance). Training requires 12GB+ VRAM and 10 to 50 sample images.

Can I run FLUX on CPU only?

Technically yes, but it is impractically slow. A single FLUX.1 [dev] image takes 10 to 20 minutes on a modern CPU versus 15 seconds on an RTX 4070. If you lack a GPU, use the DeepSeek V4 Flash approach: pay fractions of a cent per image through a cheap API rather than suffering CPU inference.