OpenLIT — OpenTelemetry LLM Observability

What it is

OpenLIT is an open-source observability platform built on OpenTelemetry that monitors LLM application costs, latency, and output quality. With a single line of code, it auto-instruments calls to OpenAI, Anthropic, Google, and other LLM providers. It also provides GPU monitoring and built-in guardrails for content safety.

It targets AI engineers and platform teams who want standardized observability for LLM applications using the OpenTelemetry ecosystem they already know.

How it saves time or tokens

OpenLIT's one-line setup eliminates the boilerplate of manually instrumenting every LLM call. By surfacing cost breakdowns per model, endpoint, and user, it identifies where tokens are being wasted. The latency tracking helps optimize slow requests, and the quality monitoring catches degraded outputs before users notice. All data flows through standard OpenTelemetry pipelines, so it integrates with existing observability stacks (Grafana, Jaeger, Datadog).

How to use

1. Install and initialize:

pip install openlit

1. Add one line to instrument all LLM calls:

import openlit

openlit.init()

# Now use any LLM library as usual
from openai import OpenAI
client = OpenAI()

response = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'Hello'}]
)
# OpenLIT automatically traces this call

1. View traces in the OpenLIT dashboard or export to any OpenTelemetry-compatible backend.

Example

import openlit
from openai import OpenAI
from anthropic import Anthropic

# Initialize once at startup
openlit.init()

# All LLM calls are automatically traced
oai = OpenAI()
ant = Anthropic()

# OpenAI call -- traced with cost, latency, tokens
oai_response = oai.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'Summarize this article'}]
)

# Anthropic call -- also traced automatically
ant_response = ant.messages.create(
    model='claude-sonnet-4-20250514',
    max_tokens=1024,
    messages=[{'role': 'user', 'content': 'Summarize this article'}]
)

# Compare costs and latency in the OpenLIT dashboard

Related on TokRepo

• AI gateway tools -- Langfuse and other LLM observability platforms
• AI tools for monitoring -- Monitoring and observability tools

Common pitfalls

• OpenLIT auto-instruments supported libraries. If you use a niche LLM provider without a supported SDK, you need manual instrumentation via OpenTelemetry spans.
• The built-in dashboard requires Docker to run. For teams already using Grafana or Datadog, export traces via OTLP instead of running a separate dashboard.
• GPU monitoring requires additional system-level permissions. Ensure the process has access to nvidia-smi or equivalent GPU metrics endpoints.