If you’re building AI applications and need to understand what’s happening inside your LLM calls — latency, cost, quality, failure modes — Langfuse is the open-source observability platform built exactly for that. Think of it as Datadog for your LLM stack: tracing, evaluation, prompt management, and cost analytics in one place.

This guide walks you through everything you need to get productive with Langfuse, from setup to advanced evaluation workflows.

What Is Langfuse?

Langfuse is an open-source LLM engineering platform. It gives you full visibility into your AI application’s behavior through structured traces, letting you debug issues, measure quality, and optimize costs. It’s the most popular open-source alternative to LangSmith.

Core capabilities:

• Tracing — Capture every LLM call, retrieval step, and tool invocation as nested spans within a trace
• Evaluations — Score outputs using LLM-as-judge, heuristic functions, or human review
• Prompt Management — Version, deploy, and A/B test prompts from a central registry
• Cost Tracking — Automatic token counting and cost calculation across models
• Datasets — Curate test sets from production data for regression testing and benchmarking

If you’re new to the broader topic, our LLM observability primer covers why this matters and what to look for in a tool.

Cloud vs Self-Hosted Setup

Langfuse offers two deployment options:

Langfuse Cloud is the fastest way to start. Sign up at cloud.langfuse.com, create a project, and grab your API keys. No infrastructure to manage.

Self-hosted gives you full data control — critical for teams with GDPR or data residency requirements. The simplest self-hosted path is Docker Compose:

# docker-compose.yml
version: "3.9"
services:
  langfuse:
    image: langfuse/langfuse:latest
    ports:
      - "3000:3000"
    environment:
      DATABASE_URL: postgresql://postgres:postgres@db:5432/langfuse
      NEXTAUTH_SECRET: your-secret-key
      NEXTAUTH_URL: http://localhost:3000
      SALT: your-salt-value
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: langfuse
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

docker compose up -d

The UI is available at http://localhost:3000. If you’re already running containers for local AI work, check out our Docker setup guide for Ollama — the networking patterns are the same.

Python SDK Integration

Install the SDK:

pip install langfuse openai

Set your credentials as environment variables or pass them directly:

import os
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com"  # or your self-hosted URL

The fastest integration path is the OpenAI drop-in wrapper. It automatically traces every call with zero code changes to your existing OpenAI usage:

from langfuse.openai import openai

# Use exactly like the normal OpenAI client — all calls are traced automatically
response = openai.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain observability in one sentence."}],
)
print(response.choices[0].message.content)

Every call now appears in your Langfuse dashboard with model, tokens, latency, and cost.

Tracing a RAG Pipeline

Real applications have multiple steps — retrieval, reranking, generation. Langfuse’s @observe() decorator captures these as nested spans automatically:

from langfuse.decorators import observe, langfuse_context
from langfuse.openai import openai

@observe()
def retrieve_documents(query: str) -> list[str]:
    # Your vector search logic here
    results = ["Doc A: Langfuse is open-source...", "Doc B: Tracing captures spans..."]
    langfuse_context.update_current_observation(
        metadata={"source": "pinecone", "top_k": 5}
    )
    return results

@observe()
def generate_answer(query: str, context: list[str]) -> str:
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": f"Answer using this context: {context}"},
            {"role": "user", "content": query},
        ],
    )
    return response.choices[0].message.content

@observe()
def rag_pipeline(query: str) -> str:
    docs = retrieve_documents(query)
    return generate_answer(query, docs)

# Run it — the full trace with nested spans appears in Langfuse
result = rag_pipeline("What is Langfuse?")

In the Langfuse UI, you’ll see a single trace for rag_pipeline containing child spans for retrieve_documents and generate_answer, each with their own timing and metadata. This is the same RAG architecture pattern you’d use with local models, just with observability baked in.

For a deeper look at how RAG fits into broader system design, see our AI app architecture patterns overview.

Evaluations

Tracing tells you what happened. Evaluations tell you how well it went. Langfuse supports three evaluation approaches:

LLM-as-Judge

Use a model to score outputs programmatically. This scales to thousands of traces:

from langfuse import Langfuse

langfuse = Langfuse()

# Score an existing trace
langfuse.score(
    trace_id="trace-abc-123",
    name="relevance",
    value=0.9,
    comment="Answer directly addresses the question using retrieved context",
)

You can automate this by fetching recent traces via the API, running them through an evaluation prompt, and posting scores back. Langfuse also has built-in eval templates in the UI for common criteria like faithfulness, relevance, and toxicity.

Human Evaluation

For high-stakes use cases, route traces to human reviewers through the Langfuse annotation queue. Reviewers see the input, output, and full trace context, then assign scores. This is especially useful for building ground-truth datasets.

Heuristic Checks

Simple programmatic checks — output length, regex patterns, JSON validity — can be posted as scores too. Combine all three approaches for a layered quality system.

Prompt Management and Versioning

Hardcoding prompts in your application code makes iteration painful. Langfuse’s prompt management lets you:

1. Create and version prompts in the UI or via API
2. Fetch the active version at runtime
3. Link traces to prompt versions automatically for A/B analysis

from langfuse import Langfuse

langfuse = Langfuse()

# Fetch the production version of a managed prompt
prompt = langfuse.get_prompt("rag-system-prompt")

# Use it in your application
compiled = prompt.compile(context="retrieved documents here")

Every trace that uses a managed prompt is automatically tagged with the prompt name and version. This means you can filter your dashboard by prompt version and compare quality metrics across iterations — no spreadsheet tracking needed.

Dashboard Overview

The Langfuse dashboard gives you a real-time view of your application’s health:

• Traces view — Browse, filter, and search all traces. Click into any trace to see the full span tree, inputs/outputs, and scores.
• Metrics — Aggregate latency (p50/p95/p99), cost per trace, token usage, and error rates over time.
• Sessions — Group traces by user session to understand multi-turn conversation flows.
• Cost analytics — Break down spending by model, feature, or user segment. Spot cost anomalies before they hit your bill.
• Score distributions — Visualize evaluation scores over time to track quality trends and catch regressions.

The dashboard is where tracing, evaluations, and prompt management come together. You can go from a cost spike → specific traces → the prompt version that caused it in a few clicks.

Langfuse vs LangSmith

Both platforms solve LLM observability, but they differ in important ways:

	Langfuse	LangSmith
Source	Open-source (MIT)	Proprietary
Self-hosting	Yes (Docker, K8s)	No
Framework lock-in	None — works with any SDK	Tightly coupled to LangChain
Prompt management	Built-in with versioning	Built-in with hub
Pricing	Free self-hosted; usage-based cloud	Usage-based cloud only
Eval framework	Flexible (LLM, human, heuristic)	Integrated with LangChain evals

Langfuse wins on openness and deployment flexibility. LangSmith wins if you’re already deep in the LangChain ecosystem and want tighter integration. For a detailed breakdown with Helicone in the mix, see our three-way comparison.

Integrations and Ecosystem

Langfuse isn’t tied to any single framework. Out of the box, it integrates with:

• OpenAI SDK — Drop-in wrapper (shown above), zero-config tracing
• LangChain / LangGraph — Callback handler that captures chain execution as traces
• LlamaIndex — Callback integration for indexing and query pipelines
• Vercel AI SDK — Trace streaming responses in Next.js applications
• LiteLLM — Proxy-level tracing across 100+ model providers
• Custom SDKs — Low-level Python and JS/TS SDKs for any framework

The LiteLLM integration is particularly useful if you’re running a model gateway. Every request routed through LiteLLM gets traced in Langfuse automatically, giving you a unified cost and quality view across OpenAI, Anthropic, local Ollama models, and anything else behind the proxy.

# LangChain integration example
from langfuse.callback import CallbackHandler

handler = CallbackHandler()

# Pass to any LangChain runnable
chain.invoke({"question": "What is Langfuse?"}, config={"callbacks": [handler]})

Tips for Production Deployments

A few things that matter once you move past prototyping:

• Sample traces in high-volume environments. Langfuse supports sampling rates so you’re not storing every single request. Set LANGFUSE_SAMPLE_RATE=0.1 to capture 10% of traffic.
• Tag traces with metadata. Add user IDs, feature flags, and environment tags so you can slice metrics meaningfully in the dashboard.
• Set up score-based alerts. Use the Langfuse API to poll for score drops and trigger alerts in Slack or PagerDuty.
• Export datasets regularly. Production traces make excellent evaluation datasets. Use the dataset feature to curate golden sets for regression testing before prompt changes.
• Use async flushing. The Python SDK batches and sends traces asynchronously by default. Call langfuse.flush() at the end of serverless function invocations to avoid dropped traces.

When to Use Langfuse

Langfuse fits best when you:

• Need full data ownership via self-hosting
• Use multiple LLM providers or frameworks (not just LangChain)
• Want prompt versioning tied directly to production traces
• Need layered evaluations combining automated and human review
• Care about cost visibility across models and features

For teams shipping AI features to production, observability isn’t optional — it’s how you maintain quality at scale. Langfuse gives you that without vendor lock-in.

• I Used It for a Week: I Used Langfuse for a Week — Open-Source LLM Observability

• AI API Pricing Compared 2026

Langfuse is one option among several. For a broader comparison, see our best monitoring tools for AI apps in 2026.

FAQ

Is Langfuse free?

Langfuse offers a generous free tier for cloud-hosted usage, and the self-hosted version is completely free and open-source. The cloud free tier includes enough traces for small projects and evaluation.

Can I self-host Langfuse?

Yes, Langfuse is fully open-source and designed for self-hosting via Docker. This gives you complete data ownership and is the recommended approach for teams with strict compliance or privacy requirements.

How does Langfuse compare to LangSmith?

Langfuse is open-source and framework-agnostic, while LangSmith is proprietary and tightly coupled to the LangChain ecosystem. Langfuse offers more deployment flexibility with self-hosting, while LangSmith provides deeper LangChain-specific integrations.

Does Langfuse work with OpenAI?

Yes, Langfuse integrates with OpenAI via its Python and JavaScript SDKs using a simple wrapper or decorator pattern. It automatically captures token usage, latency, costs, and response content from all OpenAI API calls.

---

Building a local RAG system? Start with our local RAG pipeline guide and add Langfuse tracing on top for full visibility into retrieval and generation quality.