How to Run Laguna S 2.1 Locally: Ollama, vLLM, and Hardware Guide

One of the biggest advantages of Laguna S 2.1 over proprietary models is that you can run it on your own hardware. No API costs, no rate limits, no data leaving your network. The open weights (released under OpenMDW-1.1) give you full control.

But “open weights” and “runs on my machine” are not the same thing. A 118B parameter MoE model requires planning. This guide covers exactly what hardware you need, which inference framework to use, and how to get the best performance for both Laguna S 2.1 and the smaller XS 2.1.

Hardware Requirements

Laguna S 2.1 (118B MoE, 8B Active)

The full model has 118B parameters total but only routes 8B per token through the MoE architecture. You still need to load all 118B into memory (or offload strategically), but active compute per token is modest.

Configuration	VRAM	Hardware Example	Performance
Full FP16	~240GB	3x A100 80GB or 3x H100	Full speed
FP8 quantized	~120GB	2x A100 80GB or 2x H100	Near-full quality
4-bit quantized	~60GB	1x A100 80GB or 2x RTX 4090	Good quality, some degradation
Heavy quant + offload	~24-48GB	RTX 4090 + CPU offload	Slow but functional

For most self-hosting scenarios, the sweet spot is FP8 on two high-end GPUs. You get near-full quality with manageable hardware costs.

Laguna XS 2.1 (33B MoE, 3B Active)

This is the consumer-friendly option. Released July 2, 2026, XS 2.1 is specifically designed to run on accessible hardware.

Configuration	VRAM	Hardware Example	Performance
Full FP16	~66GB	1x A100 80GB	Full speed
FP8 quantized	~33GB	1x RTX 4090 (24GB) + offload	Near-full quality
4-bit quantized	~17GB	1x RTX 4090 (24GB)	Good quality, fits easily
Heavy quant	~10GB	RTX 3080/4070	Functional, quality trade-off

With 4-bit quantization, XS 2.1 fits comfortably on a single 24GB GPU. That means RTX 4090, RTX 3090, or even RTX 4080 SUPER territory. Check our VRAM requirements guide for more detail on hardware planning.

Method 1: Ollama (Easiest)

Ollama is the fastest path from zero to running Laguna locally. It handles model downloading, quantization, and serving in one tool.

Install Ollama

curl -fsSL https://ollama.ai/install.sh | sh

Pull and Run Laguna XS 2.1

# Pull the model (auto-selects appropriate quantization)
ollama pull laguna-xs-2.1

# Run interactively
ollama run laguna-xs-2.1

# Or serve via API
ollama serve

Pull Laguna S 2.1 (requires multi-GPU)

# Pull the full model
ollama pull laguna-s-2.1

# For quantized versions
ollama pull laguna-s-2.1:q4_k_m
ollama pull laguna-s-2.1:q8_0

Ollama API Usage

Once serving, you can hit the local API at http://localhost:11434:

curl http://localhost:11434/api/generate -d '{
  "model": "laguna-xs-2.1",
  "prompt": "Write a Python function that implements binary search",
  "stream": false
}'

Ollama works well with Aider and other coding tools that support OpenAI-compatible endpoints.

Method 2: vLLM (Best for Production)

vLLM offers superior throughput for production deployments. It handles batched inference, PagedAttention for memory efficiency, and tensor parallelism across GPUs.

Install vLLM

pip install vllm

Serve Laguna S 2.1

# Basic serving with tensor parallelism across 2 GPUs
python -m vllm.entrypoints.openai.api_server \
  --model poolside/laguna-s-2.1 \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --trust-remote-code

# For 4-bit quantization
python -m vllm.entrypoints.openai.api_server \
  --model poolside/laguna-s-2.1 \
  --quantization awq \
  --tensor-parallel-size 2 \
  --max-model-len 131072

Serve Laguna XS 2.1

# Single GPU deployment
python -m vllm.entrypoints.openai.api_server \
  --model poolside/laguna-xs-2.1 \
  --max-model-len 131072 \
  --trust-remote-code

Key vLLM Configuration Options

• --tensor-parallel-size: Split model across GPUs. Use 2 for S 2.1 with FP8, 3+ for FP16.
• --max-model-len: Maximum sequence length. Reduce this if you are short on memory.
• --gpu-memory-utilization: Default 0.9. Lower if you need memory for other processes.
• --quantization: Use awq or gptq for quantized deployments.
• --enforce-eager: Disables CUDA graphs. Useful for debugging but slower.

Method 3: SGLang (Best for Complex Workflows)

SGLang excels when you need structured generation, constrained decoding, or complex multi-turn workflows.

Install SGLang

pip install sglang[all]

Serve and Use

# Launch server
python -m sglang.launch_server \
  --model-path poolside/laguna-s-2.1 \
  --tp 2 \
  --context-length 262144

# SGLang supports OpenAI-compatible API at port 30000 by default

SGLang’s RadixAttention provides efficient KV-cache reuse across requests, which is particularly valuable for iterative coding workflows where you send similar prompts with small modifications.

Optimizing Performance

Memory Management

The MoE architecture helps here. Even though the full model is 118B parameters, only 8B are active per token. This means:

• KV cache grows with sequence length, not model size
• Active compute is comparable to an 8B dense model
• Total memory is dominated by storing all expert weights

Quantization Trade-offs

For coding tasks specifically:

• FP16/BF16: Full quality. Use this if you have the VRAM.
• FP8: Minimal quality loss. The recommended default for most deployments.
• 4-bit (AWQ/GPTQ): Noticeable quality loss on complex reasoning, but still quite good for standard coding. XS 2.1 at 4-bit on a single GPU is a great daily driver.
• 2-bit: Significant degradation. Not recommended for coding tasks.

Context Length vs Memory

The 1M context window of S 2.1 requires significant KV-cache memory at full length. Practical configurations:

• 1M context: Requires dedicated GPU memory beyond model weights
• 256K context: Manageable on most multi-GPU setups
• 128K context: Comfortable on 2x 80GB GPUs with FP8
• 32K context: Works with minimal overhead

Start with a shorter max context and increase only if your tasks actually need it.

Connecting to Coding Tools

With Aider

# Point Aider at your local endpoint
aider --openai-api-base http://localhost:11434/v1 --model laguna-xs-2.1

With pool CLI

The pool CLI can connect to local deployments:

pool --endpoint http://localhost:8000 --model laguna-s-2.1

With Continue.dev or Other IDE Extensions

Most IDE extensions that support OpenAI-compatible endpoints work with local Laguna deployments. Point them at http://localhost:11434/v1 (Ollama) or http://localhost:8000/v1 (vLLM).

XS 2.1: The Consumer Sweet Spot

For most individual developers, Laguna XS 2.1 is the practical local choice. Here is why:

• 33B total params, 3B active per token
• Fits on a single 24GB GPU at 4-bit
• 262K context window (more than most models offer via API)
• 63.1% SWE-bench Multilingual (genuinely useful for real coding)
• 33.4% Terminal-Bench (good for simple agentic tasks)
• Free on OpenRouter if you want a cloud fallback

It is not going to match the full S 2.1 on complex tasks. But for autocomplete, function generation, code explanation, and simple refactoring, XS 2.1 running locally provides instant, free, private AI coding assistance.

See our best AI models for coding locally guide for how XS 2.1 compares to other local options.

Cloud vs Local: When Each Makes Sense

Run locally when:

• Privacy requirements prevent sending code to external APIs
• You need predictable latency without rate limits
• Long-term cost matters (amortized hardware is cheaper than API fees)
• You want to experiment with fine-tuning

Use the API when:

• You need 1M context (requires significant local resources for S 2.1)
• You want zero setup and maintenance
• Your usage is light enough that free tier covers it
• You need maximum throughput and Poolside handles the scaling

The beauty of open weights is that you do not have to choose permanently. Use the free API for most tasks, and deploy locally when the situation demands it.

FAQ

What is the minimum hardware to run Laguna XS 2.1?

A single GPU with 24GB VRAM (RTX 4090, RTX 3090) can run XS 2.1 at 4-bit quantization comfortably. With 16GB VRAM you would need aggressive quantization and CPU offloading, which significantly hurts speed.

Can I run the full S 2.1 on consumer hardware?

Technically yes, with heavy quantization and CPU offloading, but performance will be very slow. Realistically, S 2.1 needs at least 2x 48GB GPUs (like dual RTX 6000 Ada) for usable inference speed. For consumer-grade local use, XS 2.1 is the intended option.

Which inference framework should I use?

Ollama for simplicity and quick experimentation. vLLM for production deployments with multiple users. SGLang for complex structured generation workflows. If you are unsure, start with Ollama.

Does quantization significantly hurt coding performance?

FP8 has minimal impact (typically less than 1% benchmark degradation). 4-bit quantization shows 2-5% degradation on complex reasoning tasks but remains very capable for standard coding. For XS 2.1 on consumer hardware, 4-bit is the practical choice with good results.

How does local Laguna compare to the free OpenRouter API?

Performance is identical at the same precision and context length. The API advantage is zero setup and potentially higher throughput. The local advantage is privacy, no rate limits, and zero ongoing cost. Many developers use both depending on the situation.