SkillsMay 11, 2026·2 min read

openllmetry-js — OpenTelemetry for LLM Apps

Add OpenTelemetry traces to Node.js LLM apps with OpenLLMetry JS so you can debug latency, tool calls, and RAG steps. Works with a standard OTEL backend.

Agent Toolkit
Agent Toolkit · Community
Agent ready

Ready-to-run agent install

This asset can be installed after the agent chooses its runtime, checks the plan, and runs the matching command.

Native · 98/100Policy: allow
Agent surface
Any MCP/CLI agent
Kind
Skill
Install
Single
Trust
Trust: Established
Entrypoint
Asset
Direct install command
npx -y tokrepo@latest install 16fe280b-3881-4aae-85b8-2576cb8de8d3 --target codex

Run after dry-run confirms the install plan.

Intro

Add OpenTelemetry traces to Node.js LLM apps with OpenLLMetry JS so you can debug latency, tool calls, and RAG steps. Works with a standard OTEL backend.

  • Best for: Node.js teams who want trace-level visibility into LLM calls and tool/RAG pipelines
  • Works with: Node.js, OpenTelemetry collectors, tracing backends (Jaeger, OTEL-compatible vendors)
  • Setup time: 14 minutes

Quantitative Notes

  • Setup time ~14 minutes (npm install + OTEL exporter config)
  • GitHub stars + forks (verified): see Source & Thanks
  • Aim for <1% overhead by sampling traces (e.g., 1/100 requests in prod)

Practical Notes

Good traces answer: where time went and what failed. Model calls, retrieval, tool execution, and output parsing should each be a span. Record request IDs, model name, and high-level outcome, but avoid logging raw prompts—store hashes or redacted summaries to keep privacy and cost under control.

Safety note: Treat traces as sensitive: redact prompts, mask PII, and set retention limits.

FAQ

Q: Is this vendor-specific? A: No. It uses OpenTelemetry so you can export to any OTEL-compatible backend.

Q: What should I trace first? A: Start with LLM latency + token usage (if available), then add spans for retrieval and tools.

Q: How do I control cost? A: Use sampling, redact sensitive attributes, and keep span attributes small.

🙏

Source & Thanks

GitHub: https://github.com/traceloop/openllmetry-js Owner avatar: https://avatars.githubusercontent.com/u/125419530?v=4 License (SPDX): Apache-2.0 GitHub stars (verified via api.github.com/repos/traceloop/openllmetry-js): 398 GitHub forks (verified via api.github.com/repos/traceloop/openllmetry-js): 68

Discussion

Sign in to join the discussion.
No comments yet. Be the first to share your thoughts.

Related Assets

OpenLIT — OpenTelemetry LLM Observability

Monitor LLM costs, latency, and quality with OpenTelemetry-native tracing. GPU monitoring and guardrails built in. 2.3K+ stars.

Skills
AI Open Source

Weave — Trace and Debug LLM Apps

Weave adds tracing to LLM apps with `@weave.op`. Install `weave`, call `weave.init()`, then track inputs/outputs across API calls and validation steps.

Knowledge
Agent Toolkit

Quasar Framework — Vue.js Components for Web, Mobile, Desktop, and Electron

A high-performance Vue.js framework with 70+ components that can target SPAs, SSR, PWAs, mobile apps via Capacitor/Cordova, desktop via Electron, and browser extensions from one codebase.

Skills
AI Open Source

AdonisJS — Full-Featured MVC Framework for Node.js

AdonisJS is a TypeScript-first MVC web framework for Node.js that provides an opinionated, batteries-included experience for building server-rendered apps and APIs.

Skills
AI Open Source
◈Home🔍Search👤Me