Zep — Long-Term Memory for AI Agents and Assistants

What it is

Zep is a production memory layer for AI assistants and agents. It goes beyond simple conversation logging by extracting structured facts from dialogue, building knowledge graphs of entities and relationships, and providing temporal-aware retrieval so LLMs can recall context from hours or months ago.

Zep is designed for developers building chatbots, copilots, and autonomous agents that need to remember user preferences, prior decisions, and evolving context across sessions.

How it saves time or tokens

Without Zep, developers typically stuff entire conversation histories into prompts, burning tokens on irrelevant context. Zep's fact extraction and knowledge graph compress thousands of messages into structured memory that can be queried selectively. This reduces prompt size while improving answer relevance.

The temporal awareness layer means Zep can distinguish between what a user said last week versus what they said today, preventing stale information from polluting responses.

How to use

1. Install Zep and start the server:

pip install zep-python
# Or deploy via Docker
docker compose up -d

1. Create a session and add messages:

from zep_python import ZepClient, Message

client = ZepClient('http://localhost:8000')
session_id = 'user-123-session'

client.memory.add_memory(
    session_id,
    [
        Message(role='user', content='I prefer Python over JavaScript'),
        Message(role='assistant', content='Noted. I will suggest Python solutions.')
    ]
)

1. Retrieve relevant memory for a new prompt:

memory = client.memory.get_memory(session_id)
facts = client.memory.search_memory(session_id, 'language preference')

1. Inject the retrieved facts into your LLM prompt alongside the current user message.

Example

# Retrieve structured facts instead of raw history
results = client.memory.search_memory(
    session_id,
    query='What programming language does the user prefer?',
    limit=3
)
for r in results:
    print(f'Fact: {r.message.content} (relevance: {r.score:.2f})')

This returns only the relevant facts rather than the full conversation, keeping prompts lean.

Related on TokRepo

• AI Memory Providers — Compare Zep with Mem0, MemGPT, Letta, and other memory solutions
• AI Memory: Zep Deep Dive — Detailed Zep integration patterns and benchmarks

Common pitfalls

• Treating Zep as a simple key-value store. Its value comes from fact extraction and knowledge graphs, not raw message storage. Use the search API, not just get.
• Forgetting to set session boundaries. Without clear session IDs, memories from different users or conversations can bleed together.
• Over-relying on Zep for real-time context. Zep excels at long-term memory; for the current conversation turn, pass recent messages directly in the prompt.
• Failing to review community discussions and changelogs before upgrading. Breaking changes in major versions can disrupt existing workflows. Pin versions in production and test upgrades in staging first.