Langfuse — Open Source LLM Observability

What it is

Langfuse is an open-source LLM engineering platform that provides observability for AI applications. It traces every LLM call, tracks token usage and latency, manages prompt versions, and supports evaluation workflows. You integrate it with a few lines of code and get a dashboard showing how your AI application performs in production.

Langfuse targets AI engineers, ML teams, and product developers who build LLM-powered applications and need to understand cost, quality, and performance. It is available as a cloud service or self-hosted under the MIT license.

How it saves time or tokens

Without observability, debugging LLM applications means adding print statements, manually counting tokens, and guessing why outputs degrade. Langfuse automatically traces every call, records inputs/outputs, measures latency, and calculates costs. Prompt management lets you version and A/B test prompts without code changes. This visibility helps you identify expensive or slow calls and optimize them, directly reducing token waste.

How to use

1. Install the SDK:

pip install langfuse openai

1. Add tracing with a one-line import swap:

from langfuse.openai import openai

client = openai.OpenAI()
response = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'Hello'}]
)

1. View traces in the Langfuse dashboard at cloud.langfuse.com or your self-hosted instance.

1. Use the prompt management UI to version and deploy prompts without redeploying code.

Example

from langfuse import Langfuse

langfuse = Langfuse()

# Create a trace for a multi-step workflow
trace = langfuse.trace(name='rag-pipeline')

# Span for retrieval step
retrieval = trace.span(name='retrieval')
# ... your retrieval logic ...
retrieval.end(output={'docs_found': 5})

# Generation span for LLM call
generation = trace.generation(
    name='answer-generation',
    model='gpt-4o',
    input=[{'role': 'user', 'content': 'question'}]
)
# ... your LLM call ...
generation.end(output='answer text', usage={'input': 150, 'output': 200})

Related on TokRepo

• Langfuse on AI Gateway -- Langfuse as an observability layer
• Monitoring Tools -- Observability and monitoring tools

Common pitfalls

• Tracing adds a small latency overhead per call. For latency-sensitive applications, use async flushing (enabled by default) and batch spans.
• Self-hosted Langfuse requires PostgreSQL and ClickHouse. Plan for database maintenance and storage growth as trace volume increases.
• Prompt management works best when prompts are fetched at runtime. Hardcoded prompts in code bypass the versioning system entirely.