# PostHog LLM Observability — Track AI Agents in Production

> PostHog LLM Observability traces every LLM call from your app — model, latency, cost, errors. Auto-detects via SDK wrapper. Free up to 100K events/month.

## Install

Copy the content below into your project:

## Quick Use

1. Sign up at posthog.com → copy project API key
2. `pip install "posthog[ai]"`
3. Replace `from openai import OpenAI` with `from posthog.ai.openai import OpenAI` — pass `posthog_client=posthog`

---

## Intro

PostHog's LLM Observability traces every LLM call your app makes — model, prompt, response, latency, cost — and ties them to user sessions and feature flags. The wrapper auto-instruments OpenAI / Anthropic / LangChain / Vercel AI SDK calls; you don't change call sites. Best for: production teams who already use PostHog for product analytics and want LLM traces in the same dashboard. Works with: Python / Node SDK, OpenAI / Anthropic / LangChain. Setup time: 5 minutes.

---

### Drop in (Python)

```python
import posthog
from posthog.ai.openai import OpenAI

posthog.api_key = os.environ["POSTHOG_API_KEY"]
posthog.host = "https://us.posthog.com"

# Drop-in replacement — same API as openai.OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], posthog_client=posthog)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    posthog_distinct_id="user_42",          # tie to a user
    posthog_properties={"feature": "chat"}, # custom metadata
)
```

PostHog auto-tracks model, latency, input/output tokens, cost (using its built-in pricing table), and any error.

### Anthropic / LangChain wrappers

```python
from posthog.ai.anthropic import Anthropic
from posthog.ai.langchain import CallbackHandler

# Anthropic
client = Anthropic(api_key=..., posthog_client=posthog)

# LangChain
callbacks = [CallbackHandler(client=posthog, distinct_id="user_42")]
chain.invoke({"input": "..."}, config={"callbacks": callbacks})
```

### What you get in the dashboard

- Per-call: prompt, response, model, latency, cost, error, properties
- Aggregates: cost by model / by feature / by user, p95 latency, error rate
- Funnels: combine LLM events with normal product events ("user opened feature → 3 LLM calls → did/didn't convert")
- Cohorts: filter LLM cost by user tier, signup date, etc

### Why use PostHog instead of Helicone or Langfuse

PostHog is general product analytics with LLM as a layer. If your team already uses PostHog for funnels, retention, feature flags — adding LLM traces gives you correlation across all data. Helicone / Langfuse are LLM-specific (deeper LLM features, but no product-analytics overlap).

---

### FAQ

**Q: Is PostHog free?**
A: Yes — generous free tier (1M events/month for product analytics, 100K LLM events/month). Self-hosted is fully free under MIT license. Cloud paid tiers add longer retention and team features.

**Q: Will it slow down my LLM calls?**
A: Negligible. The wrapper sends events asynchronously in the background — your call returns the same time it would without instrumentation. PostHog buffers and batches the trace upload.

**Q: Can I exclude PII from prompts/responses?**
A: Yes — pass `posthog_privacy_mode=True` to redact prompt/response bodies (only metadata is sent), or use property scrubbing on the PostHog side. Required for healthcare / finance compliance.

---

## Source & Thanks

> Built by [PostHog](https://github.com/PostHog). Licensed under MIT.
>
> [PostHog/posthog](https://github.com/PostHog/posthog) — ⭐ 24,000+

---

<!-- ZH -->

## 快速使用

1. 在 posthog.com 注册，复制 project API key
2. `pip install "posthog[ai]"`
3. 把 `from openai import OpenAI` 换成 `from posthog.ai.openai import OpenAI`，传 `posthog_client=posthog`

---

## 简介

PostHog 的 LLM Observability 追踪你应用每次 LLM 调用 —— 模型、prompt、响应、延迟、成本 —— 并跟用户会话和 feature flag 关联。包装器自动埋点 OpenAI / Anthropic / LangChain / Vercel AI SDK 调用，调用点不用改。适合已经用 PostHog 做产品分析、想把 LLM trace 放同一仪表盘的生产团队。兼容 Python / Node SDK + OpenAI / Anthropic / LangChain。装机时间 5 分钟。

---

### 装上去（Python）

```python
import posthog
from posthog.ai.openai import OpenAI

posthog.api_key = os.environ["POSTHOG_API_KEY"]
posthog.host = "https://us.posthog.com"

# Drop-in 替换 —— API 跟 openai.OpenAI 一样
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], posthog_client=posthog)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    posthog_distinct_id="user_42",          # 关联到用户
    posthog_properties={"feature": "chat"}, # 自定义 metadata
)
```

PostHog 自动追踪模型、延迟、输入/输出 token、成本（用内置价格表）、任何错误。

### Anthropic / LangChain 包装器

```python
from posthog.ai.anthropic import Anthropic
from posthog.ai.langchain import CallbackHandler

# Anthropic
client = Anthropic(api_key=..., posthog_client=posthog)

# LangChain
callbacks = [CallbackHandler(client=posthog, distinct_id="user_42")]
chain.invoke({"input": "..."}, config={"callbacks": callbacks})
```

### 仪表盘里能看到什么

- 单次调用：prompt、响应、模型、延迟、成本、错误、属性
- 聚合：按模型 / 按功能 / 按用户分成本，p95 延迟，错误率
- 漏斗：把 LLM 事件和普通产品事件合并（「用户打开功能 → 3 次 LLM 调用 → 转化/没转化」）
- 群组：按用户档位、注册时间等过滤 LLM 成本

### 为啥用 PostHog 而不是 Helicone 或 Langfuse

PostHog 是通用产品分析，LLM 作为一层。如果你团队已经用 PostHog 做漏斗、留存、feature flag —— 加 LLM trace 让所有数据相关联。Helicone / Langfuse 是 LLM 专用（LLM 功能更深，但跟产品分析不重叠）。

---

### FAQ

**Q: PostHog 免费吗？**
A: 免费 —— 大方的免费档（产品分析每月 1M 事件，LLM 事件每月 100K）。自托管在 MIT 下完全免费。Cloud 付费档加更长保留和团队功能。

**Q: 会拖慢 LLM 调用吗？**
A: 可忽略。包装器在后台异步发事件 —— 你的调用返回时间跟没埋点一样。PostHog 缓冲并批量上传 trace。

**Q: 能从 prompt / 响应里去掉 PII 吗？**
A: 能 —— 传 `posthog_privacy_mode=True` 屏蔽 prompt/响应正文（只发 metadata），或在 PostHog 端做属性脱敏。医疗 / 金融合规需要。

---

## 来源与感谢

> Built by [PostHog](https://github.com/PostHog). Licensed under MIT.
>
> [PostHog/posthog](https://github.com/PostHog/posthog) — ⭐ 24,000+


---
Source: https://tokrepo.com/en/workflows/posthog-llm-observability-track-ai-agents-in-production
Author: PostHog