Ollama hit the context limit:

Error: prompt is too long: 8192 tokens > 4096 maximum

Here is how to fix it.

Fix 1: Reduce context size

Set a smaller context window:

# Reduce context to 2048
ollama run llama3.2 --num-ctx 2048

# For API calls
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.2",
  "prompt": "your prompt",
  "options": {"num_ctx": 2048}
}'

Fix 2: Use a model with larger context

Different models have different limits:

Model	Default context	Max context
Llama 3.2	4096	128K
Qwen 3	4096	128K
DeepSeek R1	4096	64K
Mistral	4096	32K

Fix 3: Truncate input

Cut your prompt to fit:

def truncate_prompt(prompt, max_tokens=4000):
    # Rough estimate: 1 token ≈ 4 characters
    max_chars = max_tokens * 4
    if len(prompt) > max_chars:
        return prompt[:max_chars] + "\n... (truncated)"
    return prompt

Fix 4: Summarize long context

For large documents, summarize first:

# Step 1: Summarize
summary = ollama.generate(
    model="llama3.2",
    prompt=f"Summarize in 500 words:\n{long_document}"
)

# Step 2: Use summary
response = ollama.generate(
    model="llama3.2",
    prompt=f"Based on: {summary}\n\nDo X"
)

Fix 5: Use RAG instead of full context

For large knowledge bases, use retrieval:

# Instead of sending entire codebase
# Use embeddings to find relevant chunks
relevant_chunks = search_embeddings(query, codebase_embeddings)
context = "\n".join(relevant_chunks[:5])  # Top 5 results

Fix 6: Set context in Modelfile

Create a custom model with larger context:

cat > Modelfile << 'EOF'
FROM llama3.2:latest
PARAMETER num_ctx 8192
EOF

ollama create llama3.2-8k -f Modelfile
ollama run llama3.2-8k

Still not working?

1. Use a different model — Larger context support
2. Chunk your input — Process in pieces
3. Summarize first — Reduce before sending
4. Use RAG — Retrieve only relevant content

FAQ

What is the default context length in Ollama?

The default varies by model, typically 4096 or 8192 tokens. You can check with ollama show MODEL_NAME or override it with --num-ctx flag.

Does larger context use more memory?

Yes. Context memory scales linearly. Doubling the context roughly doubles the memory overhead beyond the model weights. If you are running out of memory, reducing context can help.

Can I set context permanently?

Yes. Create a Modelfile with PARAMETER num_ctx 8192 and use ollama create to make a custom model with your preferred context size.

Related: Ollama Complete Guide · Context Window Explained · Best Long Context Models · How to Run AI Locally · Ollama Out of Memory Fix