LLM — CLI Tool for 100+ Language Models

What it is

LLM is a command-line tool and Python library by Simon Willison that provides a unified interface to 100+ language models. It works with both API-based models (OpenAI, Anthropic, Google) and local models (via plugins). Every prompt and response is automatically logged to a SQLite database for later analysis.

Developers, researchers, and power users who work with multiple LLM providers and want a single CLI to query them all will find LLM particularly useful. It eliminates the need to install separate SDKs for each provider.

How it saves time or tokens

LLM's plugin architecture means you install once and add providers as needed. The SQLite logging captures every interaction automatically, so you never lose a useful response. The embeddings subsystem lets you build semantic search over your logged conversations without a separate vector database.

How to use

1. Install LLM via pip and configure your API keys for the providers you want to use.
2. Run prompts from the command line with llm or use the Python API in scripts.
3. Query your conversation history using llm logs and the built-in SQLite database.

Example

# Install and configure
pip install llm
llm keys set openai

# Simple prompt
llm 'Explain the CAP theorem in three sentences'

# Use a specific model
llm -m claude-3.5-sonnet 'Write a Python decorator for retry logic'

# View conversation logs
llm logs -n 5

# Generate embeddings
llm embed -m ada-002 -c 'vector search query'

Related on TokRepo

• Local LLM tools — Tools for running language models locally including Ollama and LM Studio.
• AI tools for coding — CLI tools and agents for AI-assisted development.

Common pitfalls

• Not installing provider plugins. LLM ships with OpenAI support by default; other providers like Anthropic or local models require separate plugin installation via llm install llm-anthropic.
• Forgetting that logs accumulate. The SQLite database grows with every prompt. Periodically review and clean old logs if disk space is a concern.
• Assuming all models support the same features. Streaming, system prompts, and tool use vary by provider. Check the plugin documentation for model-specific capabilities.