LitGPT — Fine-Tune and Deploy AI Models Simply

What is LitGPT?

LitGPT is Lightning AI's framework for fine-tuning, pretraining, and deploying large language models. It supports 20+ model families (Llama, Mistral, Phi, Gemma, etc.) with multiple fine-tuning methods (LoRA, QLoRA, full). One-command workflows make it accessible on consumer GPUs. Built on PyTorch Lightning for scalable training.

Answer-Ready: LitGPT is Lightning AI's LLM fine-tuning and serving framework. Supports 20+ model families, LoRA/QLoRA/full fine-tuning, one-command training on consumer GPUs. Built on PyTorch Lightning. Download, fine-tune, and serve with simple CLI commands. 10k+ GitHub stars.

Best for: ML engineers fine-tuning open-source models on their own hardware. Works with: Llama, Mistral, Phi, Gemma, Falcon, and 15+ model families. Setup time: Under 5 minutes.

Core Features

1. One-Command Workflows

litgpt download <model>     # Download model
litgpt chat <model>          # Interactive chat
litgpt finetune_lora <model> # LoRA fine-tuning
litgpt finetune <model>      # Full fine-tuning
litgpt serve <model>         # API server
litgpt evaluate <model>      # Benchmark evaluation
litgpt pretrain <model>      # Pretrain from scratch

2. Supported Model Families (20+)

3. Fine-Tuning Methods

4. Training Data Format

[
  {"instruction": "Summarize this text", "input": "Long article...", "output": "Brief summary..."},
  {"instruction": "Translate to French", "input": "Hello world", "output": "Bonjour le monde"}
]

5. Multi-GPU Training

# 4 GPU training with FSDP
litgpt finetune_lora meta-llama/Llama-3.1-8B-Instruct \
  --devices 4 --strategy fsdp

FAQ

Q: Can I fine-tune on a single consumer GPU? A: Yes, QLoRA needs only 6GB VRAM. A 7B model fine-tunes on an RTX 3060.

Q: How does it compare to Unsloth? A: Unsloth is faster for single-GPU LoRA. LitGPT offers more model families, multi-GPU, and full pretraining support.

Q: Can I serve the fine-tuned model? A: Yes, litgpt serve exposes an OpenAI-compatible API endpoint.