IBM released Granite 4.1 on April 29, 2026. The headline number: the 8B dense model matches or beats the previous Granite 4.0-H-Small, a 32B Mixture-of-Experts model with 9B active parameters. That is a 4× parameter reduction with no performance loss. All three sizes — 3B, 8B, and 30B — ship under Apache 2.0, are cryptographically signed, and support up to 512K tokens of context.

This is not a reasoning model. There are no extended thinking chains, no chain-of-thought toggles. Granite 4.1 is a dense, decoder-only transformer built for predictable latency, stable token usage, and enterprise deployment. IBM trained it on approximately 15 trillion tokens across five phases and applied a four-stage reinforcement learning pipeline that caught and fixed a math regression mid-training.

Here is everything you need to know: specs, benchmarks, training details, the full model family, and how to run it.

What is IBM Granite 4.1

Granite 4.1 is IBM’s latest open-source language model family. It replaces the Granite 4.0 series with a pure dense architecture — no Mixture-of-Experts, no sparse routing. Every parameter is active on every forward pass, which means inference cost scales linearly and predictably with model size.

The family includes three sizes:

• Granite 4.1 3B — edge and mobile deployments, 128K context
• Granite 4.1 8B — the sweet spot, 512K context, fits on a single consumer GPU
• Granite 4.1 30B — maximum quality, 512K context, needs ~18 GB VRAM in FP8

Each size ships in both base and instruct variants. FP8 quantized versions are available for memory-constrained deployments.

IBM also released companion models: Granite 4.1 Vision (4B, tops Claude Opus 4.6 in table extraction), Granite 4.1 Speech (2B, 5.33% word error rate), Guardian models for safety guardrails, and embedding models covering 200+ languages.

Key specifications

Spec	3B	8B	30B
Architecture	Dense decoder-only	Dense decoder-only	Dense decoder-only
Parameters	3B	8B	30B
Context window	128K	512K	512K
Training tokens	~15T (shared pipeline)	~15T	~15T
License	Apache 2.0	Apache 2.0	Apache 2.0
Quantization	FP8 available	FP8 available	FP8 available
Cryptographic signing	Yes	Yes	Yes
ISO certified	Yes	Yes	Yes

All models use the same five-phase training pipeline and four-stage RL alignment. The 3B model caps at 128K context; the 8B and 30B extend to 512K through staged context extension with model merging.

Benchmark results

The instruct variants tell the real story. Here is how the three sizes compare across major benchmarks:

Benchmark	3B	8B	30B
MMLU (5-shot)	67.02	73.84	80.16
IFEval Avg	82.3	87.06	89.65
ArenaHard	37.8	68.98	71.02
GSM8K (8-shot)	86.88	92.49	94.16
HumanEval (pass@1)	79.27	87.2	89.63
BFCL V3 (tool calling)	60.8	68.27	73.68
EvalPlus (coding)	67.1	80.2	82.7
MMLU-Pro	49.8	56.0	64.1
MMMLU (5-shot)	57.61	64.84	73.71
AttaQ (safety)	81.88	81.19	85.76

The 8B model hits 87.2 on HumanEval, 92.49 on GSM8K, and 80.2 on EvalPlus. These numbers match or exceed the previous Granite 4.0-H-Small (32B MoE, 9B active parameters) across most benchmarks. For a dense 8B model, that is exceptional.

The 30B model leads BFCL V3 tool calling at 73.68, ahead of Gemma-4-31B at 72.7. If your workload involves function calling and tool use, the 30B is currently the best open-source option in its weight class.

Long-context performance on the RULER benchmark (8B base model):

Context length	Score
32K	83.6
64K	79.1
128K	73.0

The 30B base scores 85.2, 84.6, and 76.7 at the same context lengths. Performance degrades gracefully rather than collapsing at longer contexts — a direct result of IBM’s staged context extension approach.

How IBM trained Granite 4.1

Five-phase training pipeline

IBM used a progressive data annealing strategy across five training phases:

1. Phase 1 — Broad web and code data. General knowledge acquisition.
2. Phase 2 — Higher-quality filtered data. Domain-specific knowledge deepening.
3. Phase 3 — Instruction-following and task-specific data.
4. Phase 4 — Short-context fine-tuning and alignment.
5. Phase 5 — Long-context extension (32K → 128K → 512K).

The total training corpus spans approximately 15 trillion tokens. Before fine-tuning, IBM applied LLM-as-Judge data filtering: a six-dimension scoring system that automatically rejects samples containing hallucinations. This is not just quality filtering — it is a systematic approach to preventing the model from learning to hallucinate during supervised fine-tuning.

Four-stage reinforcement learning

After supervised fine-tuning, IBM applied four stages of RL:

1. Joint multi-domain RL — Broad capability improvement across reasoning, coding, and general tasks.
2. RLHF for chat — Human preference alignment for conversational quality.
3. Identity and knowledge calibration — Ensuring the model knows what it knows and does not fabricate.
4. Math recovery RL — IBM detected that RLHF caused a regression in mathematical reasoning. They added a dedicated RL stage to recover math performance without losing chat quality.

That fourth stage is notable. Most model developers either do not detect post-RLHF regressions or accept them as a tradeoff. IBM caught the math regression and fixed it with targeted RL, which suggests a mature evaluation pipeline.

Staged context extension

Extending context from 32K to 512K without destroying short-context performance is hard. IBM used a staged approach:

• Train at 32K context first
• Extend to 128K with continued training
• Extend to 512K with final training phase
• Apply model merging to preserve short-context quality

The model merging step is key. Without it, long-context training typically degrades performance on shorter inputs. IBM’s approach maintains strong RULER scores at 32K while still handling 512K contexts.

Granite 4.1 vs the competition

How does Granite 4.1 stack up against other open-source models in April 2026?

Model	Size	Architecture	Context	HumanEval	BFCL V3	License
Granite 4.1 8B	8B	Dense	512K	87.2	68.27	Apache 2.0
Granite 4.1 30B	30B	Dense	512K	89.63	73.68	Apache 2.0
Qwen3.6 35B-A3B	35B (3B active)	MoE	128K	—	—	Apache 2.0
Gemma 4 31B	31B	Dense	128K	—	72.7	Permissive
Devstral Small 24B	24B	Dense	256K	—	—	Apache 2.0
Llama 4 Scout	—	MoE	128K	—	—	Llama License

Granite 4.1’s advantages are clear: 512K context (2–4× competitors), leading tool-calling scores, and Apache 2.0 licensing. The 8B model’s efficiency is unmatched — you get 87.2 HumanEval in a model that fits in 5 GB of VRAM.

The tradeoff: Granite 4.1 is not a reasoning model. If you need extended chain-of-thought for complex multi-step problems, models like Qwen3.6 with thinking mode or dedicated reasoning models will outperform it. Granite 4.1 is built for fast, predictable inference — not deep deliberation.

For a broader comparison of coding models you can run locally, see our guide to the best Ollama models for coding in 2026.

The full Granite 4.1 model family

Granite 4.1 is not just language models. IBM released a complete ecosystem:

Language models — 3B, 8B, 30B (base + instruct + FP8 for each)

Vision model — Granite 4.1 Vision 4B. Scores 86.5 on table extraction, beating Claude Opus 4.6 at 83.8. Useful for document processing, OCR, and structured data extraction from images.

Speech model — Granite 4.1 Speech 2B. Achieves 5.33% word error rate. Designed for enterprise transcription workloads.

Guardian models — Safety guardrails that run alongside the main model. Integrated with IBM’s AI Risk Atlas for enterprise compliance. These are separate models that evaluate inputs and outputs for safety, bias, and policy compliance.

Embedding models — Support 200+ languages. Useful for RAG pipelines and semantic search.

This breadth matters for enterprise deployments. You can build a complete AI pipeline — speech-to-text, document understanding, language generation, safety filtering — entirely within the Granite ecosystem, all under Apache 2.0.

How to run Granite 4.1

Ollama

The fastest way to get started:

# Pull the 8B instruct model
ollama pull granite4.1:8b

# Run it
ollama run granite4.1:8b

# Or pull the 3B for lighter hardware
ollama pull granite4.1:3b

The 3B runs comfortably on consumer hardware with 4 GB of RAM. The 8B needs about 5 GB of VRAM. For a comparison of local inference engines, see our Ollama vs llama.cpp vs vLLM breakdown.

vLLM

For production serving with high throughput:

pip install vllm

vllm serve ibm-granite/granite-4.1-8b-instruct \
  --max-model-len 131072 \
  --tensor-parallel-size 1

vLLM supports Granite 4.1 out of the box. Use --tensor-parallel-size to shard across multiple GPUs for the 30B model.

HuggingFace Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "ibm-granite/granite-4.1-8b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

messages = [{"role": "user", "content": "Explain the difference between dense and MoE architectures."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

watsonx.ai

IBM’s managed cloud platform offers Granite 4.1 with enterprise SLAs, monitoring, and governance tools. If you are already in the IBM ecosystem, this is the path of least resistance.

Other platforms

Granite 4.1 is also available on OpenRouter, Replicate, LM Studio, and AnythingLLM. For running the 30B model without local hardware, see our guide on how to run Mistral Medium 3.5 locally — the same cloud GPU strategies apply.

What changed from Granite 4.0

The jump from 4.0 to 4.1 is significant:

Architecture shift — Granite 4.0 used MoE for its larger models (4.0-H-Small was 32B MoE with 9B active). Granite 4.1 is entirely dense. This simplifies deployment, reduces inference complexity, and makes performance more predictable.

Efficiency gains — The 8B dense model matches the 32B MoE predecessor. IBM achieved this through better training data, the five-phase pipeline, and the four-stage RL process.

Context window — 4.0 models topped out at 128K. The 4.1 8B and 30B models extend to 512K through staged context extension with model merging.

Training scale — ~15 trillion tokens across five phases, up from previous generations.

Enterprise trust — Cryptographic signing and ISO certification are new additions. Combined with Guardian models and the AI Risk Atlas integration, Granite 4.1 is the most enterprise-ready open-source model family available.

Model family breadth — Vision, Speech, Guardian, and Embedding models round out the ecosystem. Previous Granite releases were primarily language-focused.

When to use Granite 4.1

Pick the 3B when you need edge deployment, mobile inference, or a lightweight assistant. It scores 79.27 on HumanEval and 86.88 on GSM8K — strong for its size. The 128K context is sufficient for most single-document tasks.

Pick the 8B for the best balance of quality and efficiency. It fits on any modern GPU, handles 512K context, and matches models 4× its size. This is the default choice for most developers.

Pick the 30B when you need maximum quality, especially for tool calling (73.68 BFCL V3) or complex coding tasks (82.7 EvalPlus). Budget ~18 GB VRAM with FP8 quantization.

Skip Granite 4.1 if you need deep reasoning chains, extended thinking, or state-of-the-art agentic coding (SWE-bench). Granite 4.1 is optimized for fast, predictable inference — not multi-step deliberation.

FAQ

Is Granite 4.1 free to use commercially?

Yes. All Granite 4.1 models are released under Apache 2.0, which allows unrestricted commercial use, modification, and redistribution. There are no usage caps, no registration requirements for the weights, and no restrictive clauses. You can fine-tune, deploy, and sell products built on Granite 4.1 without paying IBM anything.

How does the 8B model beat the previous 32B MoE?

Better training, not more parameters. IBM used a five-phase training pipeline on 15 trillion tokens with progressive data annealing, LLM-as-Judge data filtering that rejects hallucinated training samples, and a four-stage RL pipeline. The combination of higher-quality data, better filtering, and more sophisticated alignment produces a dense 8B model that matches the 32B MoE (9B active parameters) from the previous generation. Dense architectures also benefit from every parameter being active on every token, whereas MoE models only route to a subset of experts.

What hardware do I need to run Granite 4.1?

The 3B model runs on any machine with 4 GB of RAM — laptops, Raspberry Pi-class devices, phones. The 8B model needs about 5 GB of VRAM, which fits on an RTX 3060 or any Apple Silicon Mac. The 30B model requires approximately 18 GB of VRAM with FP8 quantization, fitting on an RTX 4090 or a Mac with 32 GB unified memory. For detailed hardware guidance, check our VRAM requirements guide.

Does Granite 4.1 support function calling and tool use?

Yes. The instruct variants support function calling natively. The 30B model leads BFCL V3 at 73.68, ahead of Gemma-4-31B (72.7). The 8B scores 68.27 on the same benchmark. Function calling works through the standard tool-use chat template — define your tools in the system message, and the model will generate structured function calls in response to user queries.

How does the 512K context window compare to competitors?

Granite 4.1’s 512K context is 2–4× larger than most competitors. Qwen3.6 offers 128K, Gemma 4 offers 128K, and Devstral Small provides 256K. The 512K window lets you process entire codebases, long documents, or extended conversation histories in a single pass. IBM’s staged extension approach (32K → 128K → 512K with model merging) preserves short-context quality, so you do not pay a performance penalty on shorter inputs.

What is the difference between Granite 4.1 base and instruct models?

Base models are pre-trained on raw text and code — they complete text but do not follow instructions. Instruct models are fine-tuned with supervised instruction data and aligned with RLHF. For most use cases (chat, coding assistance, tool calling, API serving), you want the instruct variant. Use base models only if you plan to do your own fine-tuning or need raw text completion.