Kimi K3 open weights dropped on July 27, 2026. The download is 1.56 TB across 96 shards in MXFP4 format. It is the largest open-weight model ever released to the public.
Here is the honest reality: you cannot run Kimi K3 on consumer hardware. Not on a Mac Studio. Not on a gaming PC. Not on a multi-GPU workstation. The model requires a datacenter GPU cluster. This guide explains why, what you actually need, and what your practical alternatives are.
If you just want to use Kimi K3 without self-hosting, the API costs $3/$15 per million tokens. See our Kimi K3 complete guide for API setup.
Why Kimi K3 Cannot Run on Consumer Hardware
Kimi K3 uses a Mixture-of-Experts architecture with 896 experts. During inference, only 16 experts are active per token. But here is the critical detail: all 896 experts must be loaded into memory simultaneously. The router needs to access any expert at any token, so you cannot offload unused experts to disk.
This means:
- Full precision (BF16): ~5.6 TB memory needed
- MXFP4 (released format): ~1.56 TB memory needed
- Aggressive 2-bit quantization: ~870 GB minimum
- Even 1-bit quantization: ~500+ GB
No consumer device has this much memory:
- Mac Studio Ultra maxes at 512 GB (not enough even for 2-bit)
- RTX 4090 has 24 GB (you would need 65+ of them)
- A100 80 GB (you need 20+ of them)
- H100 80 GB (you need 20+ of them minimum)
Hardware Requirements
Minimum viable configuration
| Component | Requirement |
|---|---|
| GPUs | 8x H100 80GB (640 GB VRAM) with aggressive quantization |
| Alternative | 16x A100 80GB (1.28 TB VRAM) for comfortable MXFP4 |
| Interconnect | NVLink or InfiniBand (required for multi-GPU inference) |
| System RAM | 512 GB+ (for model loading and KV cache) |
| Storage | 2+ TB NVMe SSD (for model files) |
| Network | 100 Gbps+ between nodes if multi-node |
Recommended production configuration
Moonshot recommends 64+ accelerators for production serving. Their own infrastructure uses:
- Multi-node H100 clusters with InfiniBand
- Custom tensor parallelism across nodes
- Optimized MoE routing for the 896-expert architecture
Cost estimate for self-hosting
| Setup | Monthly cost (cloud) | Speed |
|---|---|---|
| 8x H100 (RunPod, on-demand) | ~$25,000/month | ~15-30 tok/s |
| 16x A100 (AWS p4d.24xlarge) | ~$35,000/month | ~30-50 tok/s |
| 64x H100 (production) | ~$150,000+/month | ~80+ tok/s |
Compare this to the API at $3/$15 per million tokens. At $25,000/month self-hosting cost, you would need to process roughly 1.5 billion tokens per month just to break even with the API. For most organizations, the API is dramatically cheaper.
How to Download the Weights
The weights are on HuggingFace under moonshotai/Kimi-K3:
# Install dependencies
pip install huggingface_hub hf_transfer
# Enable fast downloads
export HF_HUB_ENABLE_HF_TRANSFER=1
# Download all 96 shards (1.56 TB total)
huggingface-cli download moonshotai/Kimi-K3 \
--local-dir ./kimi-k3-weights/ \
--resume-download
# This will take several hours even on a fast connection
# At 1 Gbps: ~3.5 hours minimum
# At 10 Gbps: ~20 minutesThe download contains:
- 96 weight shard files (MXFP4 quantization)
- Configuration files (model architecture)
- Tokenizer files
- Technical report
Running with vLLM (Multi-GPU)
If you have the hardware, vLLM with tensor parallelism is the recommended inference engine:
pip install vllm
# Serve on 16x A100 80GB with tensor parallelism
python -m vllm.entrypoints.openai.api_server \
--model ./kimi-k3-weights/ \
--tensor-parallel-size 16 \
--max-model-len 131072 \
--gpu-memory-utilization 0.95 \
--trust-remote-code \
--port 8000This gives you an OpenAI-compatible API running on your own infrastructure.
With SGLang (alternative)
pip install sglang
python -m sglang.launch_server \
--model-path ./kimi-k3-weights/ \
--tp 16 \
--mem-fraction-static 0.9 \
--port 8000Running on Cloud GPU Providers
The most practical “self-hosted” path for most teams:
RunPod
# Create a pod with 8x H100 80GB
# Select the "8x H100 SXM" template
# Cost: ~$33/hour on-demand, ~$20/hour spot
# SSH in and download weights
pip install huggingface_hub hf_transfer
HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download moonshotai/Kimi-K3 \
--local-dir /workspace/kimi-k3/
# Start serving
pip install vllm
python -m vllm.entrypoints.openai.api_server \
--model /workspace/kimi-k3/ \
--tensor-parallel-size 8 \
--max-model-len 65536 \
--port 8000AWS (p5.48xlarge - 8x H100)
# ~$98/hour on-demand
# Use Deep Learning AMI with pre-installed CUDA
# Download and serve
huggingface-cli download moonshotai/Kimi-K3 --local-dir /opt/kimi-k3/
python -m vllm.entrypoints.openai.api_server \
--model /opt/kimi-k3/ \
--tensor-parallel-size 8 \
--max-model-len 65536 \
--port 8000License
Kimi K3 uses an MIT-derived license with revenue-tiered commercial terms:
- Free for personal, academic, and research use
- Free commercial use for organizations under a revenue threshold
- Commercial license required above the threshold (contact Moonshot)
- No restriction on model modification, fine-tuning, or derivative works
This is more permissive than Meta’s Llama license (700M MAU cap) but less permissive than pure MIT (like DeepSeek V4 Pro).
Practical Alternatives
For most developers, running K3 locally makes no sense. Here are your real options:
| Option | Cost | Speed | Effort |
|---|---|---|---|
| Kimi K3 API | $3/$15 per M tokens | ~80 tok/s | Minutes |
| K3 via OpenRouter | ~$3.50/$17 per M tokens | Varies | Minutes |
| Self-host on RunPod | ~$33/hour | ~20-30 tok/s | Hours |
| Self-host on AWS | ~$98/hour | ~30-50 tok/s | Hours |
For comparison, these models offer similar quality on consumer hardware:
- DeepSeek V4 Pro (1.6T MoE, fits on 4x A100, self-hostable)
- Qwen 3.6-27B (27B dense, runs on 16GB)
- Poolside Laguna S 2.1 (118B MoE, runs on 1 GPU)
When Self-Hosting Kimi K3 Makes Sense
Self-hosting K3 is justified only if you meet ALL of these criteria:
- You process 1.5+ billion tokens per month (break-even with API)
- You need data sovereignty (tokens never leave your network)
- You have existing GPU infrastructure (already paying for H100s)
- You need custom fine-tuning or model modification
- You need guaranteed latency with no rate limits
For everyone else: use the $3/$15 API. It is 80 tok/s, has 1M context, and requires zero infrastructure.
FAQ
Can I run Kimi K3 on a Mac Studio?
No. Even the maxed-out Mac Studio M4 Ultra with 512 GB unified memory cannot fit Kimi K3. The model requires at minimum ~870 GB for aggressive 2-bit quantization, and all 896 experts must be in memory simultaneously. There is no workaround for this.
Can I run Kimi K3 on a single GPU?
No. Even an H100 with 80 GB VRAM is far too small. You need a minimum of 8x H100 80 GB (640 GB total VRAM) with aggressive quantization, or 16+ GPUs for comfortable inference.
What is the cheapest way to use Kimi K3?
The API at $3/$15 per million tokens. Self-hosting costs $25,000+/month in cloud GPU rental. The API is cheaper unless you process over 1.5 billion tokens per month.
How does the Kimi K3 license work?
MIT-derived with revenue-tiered commercial terms. Free for personal, academic, research, and most commercial use. Organizations above a revenue threshold need a commercial license from Moonshot. No restrictions on modification or fine-tuning.
Is Kimi K3 the largest open-weight model?
Yes. At 2.8 trillion parameters across 896 experts, it is the largest model ever released with open weights. The next largest is DeepSeek V4 Pro at 1.6T (also MoE).
Should I wait for quantized versions?
Community GGUF quantizations already exist (audnai on HuggingFace). However, even aggressive quantization cannot bring K3 below ~500 GB, which still exceeds all consumer hardware. Quantized versions help reduce datacenter costs (fewer GPUs needed) but do not enable consumer use.