Letta — AI Agent Long-Term Memory Framework

What it is

Letta, formerly known as MemGPT, is an open-source framework for building AI agents that maintain persistent long-term memory across conversations. It addresses the fundamental context window limitation of LLMs by implementing a tiered memory architecture inspired by operating system virtual memory.

The framework is aimed at developers building stateful LLM applications -- chatbots that remember users, research agents that accumulate knowledge, and assistants that improve over time without losing context.

How it saves time or tokens

Without Letta, developers must manually manage context windows: truncating old messages, summarizing history, or re-injecting relevant facts. Letta automates this entirely. The agent decides what to store in core memory (always in context), recall memory (searchable conversation history), or archival memory (unlimited vector-indexed storage). This reduces wasted tokens on context management boilerplate and avoids the common failure mode where agents forget critical user preferences mid-conversation.

The workflow estimates around 4,100 tokens for a basic setup, but the real savings come from eliminating repeated context injection across sessions.

How to use

1. Install Letta and start the server:

pip install letta
letta server

1. Create a client and configure an agent with memory:

from letta import create_client

client = create_client()
agent = client.create_agent(
    name='my_agent',
    memory=client.create_block(
        'You are a helpful assistant.',
        label='system'
    ),
)

1. Send messages and let the agent manage its own memory:

response = agent.send_message(
    'Remember: my favorite color is blue.'
)
print(response.messages)

Example

from letta import create_client

client = create_client()

# Create agent with tiered memory
agent = client.create_agent(
    name='research_assistant',
    memory=client.create_block(
        'You are a research assistant that remembers '
        'all papers the user has discussed.',
        label='system'
    ),
)

# Agent stores facts in archival memory automatically
agent.send_message('I just read the Attention Is All You Need paper.')
agent.send_message('What papers have I mentioned?')

Related on TokRepo

• AI memory frameworks compared -- Browse all memory solutions including Mem0, Zep, and MemGPT variants
• Letta deep-dive on TokRepo -- Dedicated page for Letta architecture and usage patterns
• Agent tools directory -- Other frameworks for building autonomous AI agents

Common pitfalls

• Running letta server without sufficient disk space for the SQLite-backed archival memory can cause silent failures on large datasets
• The tiered memory system works best with models that follow function-calling conventions; smaller open models may not reliably trigger memory operations
• Confusing Letta (the rebranded project) with the original MemGPT academic paper -- the API surface has changed substantially since the rename