Text Generation WebUI — Local LLM Chat Interface

What it is

Text Generation WebUI (by oobabooga) is a Gradio-based web interface for running large language models locally. It supports multiple backends including llama.cpp, ExLlamaV2, Transformers, and AutoGPTQ. Features include text chat, instruction-following, vision (image input), LoRA training, image generation, and an OpenAI-compatible API server. Everything runs on your hardware with no data leaving your machine.

This tool is for developers and AI enthusiasts who want to run open-source LLMs locally for privacy, experimentation, or offline use. If you want a ChatGPT-like interface for models like LLaMA, Mistral, or Qwen without cloud dependencies, this is the standard choice.

How it saves time or tokens

Text Generation WebUI provides a unified interface for multiple model backends, eliminating the need to set up separate environments for each. The one-click installer handles Python dependencies, CUDA drivers, and model downloading. The OpenAI-compatible API means existing tools and scripts that target OpenAI can point to your local instance without code changes. Running locally also means zero API costs -- no token billing.

How to use

1. Download the one-click installer from the GitHub releases page for your OS (Windows, macOS, Linux).
2. Run the installer, which sets up a Python environment with all dependencies.
3. Download a model from HuggingFace through the UI or place model files in the models directory.

Example

# Manual installation
git clone https://github.com/oobabooga/text-generation-webui.git
cd text-generation-webui
pip install -r requirements.txt

# Start the web UI
python server.py

# Start with OpenAI-compatible API
python server.py --api --listen

# The API is now available at http://localhost:5000/v1
# Use it with any OpenAI client:
curl http://localhost:5000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "local-model", "messages": [{"role": "user", "content": "Hello"}]}'

Related on TokRepo

• Local LLM tools -- Text Generation WebUI deep dive
• Self-hosted AI tools -- run AI tools on your own infrastructure

Common pitfalls

• Not having enough VRAM for the chosen model. A 7B parameter model needs roughly 6GB VRAM in 4-bit quantization. Check model requirements before downloading.
• Using the wrong backend for your hardware. llama.cpp works well on CPU and Apple Silicon. ExLlamaV2 is optimized for NVIDIA GPUs. The UI lets you switch backends in settings.
• Running the API server without authentication on a public network. The OpenAI-compatible API has no built-in auth. Use a reverse proxy or firewall rules if exposing it beyond localhost.