Opik — Debug, Evaluate & Monitor LLM Apps

What it is

Opik is an open-source LLM observability platform by Comet that provides tracing, evaluation, and production monitoring for AI applications. It instruments LLM calls with a single decorator, runs automated quality evaluations on your outputs, and monitors RAG retrieval quality and agent behavior in production.

It targets AI engineers building production LLM applications who need to debug issues, measure output quality systematically, and catch regressions before users report them.

How it saves time or tokens

Opik surfaces the root cause of quality issues faster than manual debugging. The tracing view shows exactly which step in a multi-step chain produced a bad output, with token counts, latency, and cost for each step. Automated evaluations run continuously, so you know when prompt changes improve or degrade quality. For RAG applications, Opik evaluates retrieval relevance and generation faithfulness, identifying where tokens are wasted on irrelevant context.

How to use

1. Install and configure:

pip install opik
opik configure

1. Add tracing with one decorator:

import opik

@opik.track()
def generate_answer(question: str):
    from openai import OpenAI
    client = OpenAI()
    response = client.chat.completions.create(
        model='gpt-4o',
        messages=[{'role': 'user', 'content': question}]
    )
    return response.choices[0].message.content

1. Run evaluations on your dataset:

from opik.evaluation import evaluate
from opik.evaluation.metrics import Hallucination, AnswerRelevance

results = evaluate(
    experiment_name='qa-v2',
    dataset=my_dataset,
    task=generate_answer,
    scoring_metrics=[Hallucination(), AnswerRelevance()]
)

Example

import opik
from opik.evaluation import evaluate
from opik.evaluation.metrics import Hallucination, AnswerRelevance

@opik.track()
def rag_pipeline(query: str):
    # Retrieve relevant documents
    docs = vector_store.similarity_search(query, k=3)
    context = '\n'.join([d.page_content for d in docs])
    
    # Generate answer with context
    from openai import OpenAI
    client = OpenAI()
    response = client.chat.completions.create(
        model='gpt-4o',
        messages=[
            {'role': 'system', 'content': f'Context: {context}'},
            {'role': 'user', 'content': query}
        ]
    )
    return response.choices[0].message.content

# Evaluate the RAG pipeline
results = evaluate(
    experiment_name='rag-eval-v1',
    dataset=test_questions,
    task=rag_pipeline,
    scoring_metrics=[Hallucination(), AnswerRelevance()]
)

Related on TokRepo

• AI tools for monitoring -- LLM monitoring and observability platforms
• AI tools for testing -- Testing and evaluation frameworks for AI

Common pitfalls

• Evaluation metrics like Hallucination use an LLM judge, which adds API costs. Run evaluations on representative samples rather than entire datasets to control costs.
• Tracing in production generates significant data volume. Configure sampling rates for high-traffic applications to keep storage and costs manageable.
• Custom metrics require understanding of the scoring API. Start with built-in metrics (Hallucination, AnswerRelevance, Moderation) before writing custom evaluators.