LLM Observability
Traceloop — OpenTelemetry-first LLM Observability logo

Traceloop — OpenTelemetry-first LLM Observability

Traceloop ships OpenLLMetry, the popular OSS library for instrumenting LLM apps with OpenTelemetry. Backend-agnostic traces: send to Traceloop Cloud, Grafana, Datadog, or your existing OTEL stack.

Why Traceloop

Traceloop’s position is "OTEL all the way". Their open-source OpenLLMetry library is SDK-like — one call at startup (Traceloop.init()) auto-instruments the LLM clients and frameworks you’re already using (OpenAI, Anthropic, LangChain, LlamaIndex, Pinecone, Weaviate, Chroma, Qdrant). All spans use the OpenTelemetry GenAI semantic conventions.

Backend flexibility is the real value. Your existing observability stack — Grafana Tempo, Datadog APM, Jaeger, New Relic, Honeycomb, Dynatrace — already understands OTEL traces. With Traceloop you get LLM-specific spans into those backends without standing up a separate LLM-observability platform. Traceloop Cloud is a managed LLM-focused backend option, but it’s not required.

Against Langfuse / Phoenix: Traceloop is thinner on the backend side (no prompt registry, lighter eval story) but stronger on "works with whatever APM you already run". For organizations that have already standardized on Datadog or Grafana, Traceloop is the LLM-aware layer that keeps everything in one observability plane.

Quick Start — One-line Init

Traceloop.init() wires up OpenTelemetry and auto-instruments every supported library it finds in the Python path. The @workflow and @task decorators create explicit spans — useful for naming the steps the way your team thinks about them, so traces read like operations not library internals.

# pip install traceloop-sdk
from traceloop.sdk import Traceloop
from traceloop.sdk.decorators import workflow, task

# Point at whatever OTEL backend you prefer
# (Traceloop Cloud, Jaeger, Grafana Tempo, Datadog, Honeycomb, ...)
Traceloop.init(
    app_name="tokrepo-demo",
    api_endpoint="https://api.traceloop.com",  # or your own OTEL collector URL
    api_key="tl-...",                            # or None for self-hosted
)

# Auto-instruments: OpenAI, Anthropic, LangChain, LlamaIndex, Pinecone, Chroma, ...
from openai import OpenAI
client = OpenAI()

@task(name="answer_question")
def answer(q: str) -> str:
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": q}],
    )
    return resp.choices[0].message.content

@workflow(name="qa_chain")
def qa_chain(user_q: str) -> str:
    # Nested spans: qa_chain → answer_question → OpenAI LLM call
    return answer(user_q)

print(qa_chain("Why does OpenTelemetry matter for LLMs?"))
# Backend now shows a trace tree with latency/cost/prompt at every level.

Key Features

OpenLLMetry auto-instrumentation

One init() call auto-instruments 20+ LLM and vector DB libraries. Uses OpenTelemetry GenAI semantic conventions — future-proof against changes in any specific vendor.

Backend-agnostic

Ship traces to Traceloop Cloud, Grafana Tempo, Datadog, Jaeger, New Relic, Honeycomb, SigNoz, or any OTEL-compatible backend. Your APM choice, not Traceloop’s.

Python + TypeScript SDKs

First-class support for both major agent ecosystems. Same semantic conventions on both, so traces from a Python backend + TS frontend agent interleave cleanly.

Prompt versioning (Traceloop Cloud)

Cloud tier adds a prompt registry with versioning and deployment labels — optional, not required for the OSS instrumentation.

LLM-as-judge evals (OSS)

OpenLLMetry ships a small eval package — fewer pre-built evaluators than Phoenix, but covers the common ones (faithfulness, relevance) and runs anywhere.

Standards alignment

Active contributor to the OpenTelemetry GenAI semantic conventions. If you expect the ecosystem to converge on OTEL for LLM observability, Traceloop is in the middle of that standardization.

Comparison

 Backend ChoiceInstrumentationPrompt RegistryBest For
TraceloopthisAny OTEL backendAuto via OpenLLMetryVia Traceloop CloudTeams with existing APM
LangfuseLangfuse onlySDK + OTEL ingestFirst-classLLM-specific ops
Arize PhoenixPhoenix onlyOpenInference OTELVia playgroundEvals + research
HeliconeHelicone onlyProxy-basedYesZero-code speed

Use Cases

01. Organizations with existing APM

You already run Datadog, Grafana, or New Relic. Traceloop adds LLM-aware spans into the same backend — single pane of glass for service + LLM traces.

02. Polyglot stacks

Python backend, TS frontend, Go services — all instrumented with OTEL. Traceloop’s OpenLLMetry keeps LLM spans consistent across languages, where Langfuse/Phoenix SDKs are per-language.

03. Standards-oriented teams

Teams that avoid vendor-specific trace formats on principle. OTEL GenAI conventions give you portability if you ever swap observability vendors.

Pricing & License

OpenLLMetry (OSS SDK): Apache 2.0. Free forever. Use with any OTEL backend — zero Traceloop cost if you’re sending to your own infra.

Traceloop Cloud: managed LLM-specific backend with prompt registry, dashboards, and evals. Free tier for dev, usage-based paid plans. See traceloop.com/pricing.

Hybrid deployment: many teams send copies of traces to both their APM (Datadog) and Traceloop Cloud. OTEL supports multiple exporters — pay only for what each backend actually stores.

Frequently Asked Questions

Traceloop vs Langfuse?+

Traceloop is backend-agnostic — send traces to your existing APM. Langfuse is a purpose-built LLM backend with richer product features (prompt mgmt, datasets, evals). Traceloop for OTEL-native orgs; Langfuse for LLM-specific ops workflows.

Do I have to use Traceloop Cloud?+

No. OpenLLMetry is a standalone OSS SDK — point it at any OTEL-compatible backend (Grafana, Datadog, Jaeger, Honeycomb, self-hosted Tempo). Traceloop Cloud is a convenience, not a lock-in.

Which frameworks does OpenLLMetry auto-instrument?+

As of 2026: OpenAI, Anthropic, Google Gemini, Cohere, HuggingFace, Replicate, Mistral, Bedrock, Vertex, LangChain, LlamaIndex, CrewAI, Haystack, DSPy, Pinecone, Chroma, Weaviate, Qdrant, pgvector, Milvus, and more. See the OpenLLMetry GitHub README for the current list.

Does it work with Datadog LLM Observability?+

Yes — Datadog ingests OTEL traces and has dedicated LLM views. Traceloop + Datadog is a common combo for teams already on Datadog APM.

Is the prompt registry only in Traceloop Cloud?+

Yes. The OSS SDK focuses on instrumentation. If you need versioned prompt storage with deployment labels, you either use Traceloop Cloud or pair OpenLLMetry with a prompt-focused tool like Langfuse or Portkey.

Compare Alternatives