Unsloth — 2x Faster Local LLM Training & Inference

What it is

Unsloth is a unified local interface for running and training AI models. It provides up to 2x faster training with 70% less VRAM usage across 500+ models including Qwen, DeepSeek, Llama, and Gemma. It includes a web UI with one-click fine-tuning, a CLI for automated workflows, and full compatibility with the Hugging Face ecosystem.

It targets ML engineers and developers who want to fine-tune LLMs on consumer GPUs without expensive cloud compute, and researchers who need faster iteration cycles.

How it saves time or tokens

Unsloth's memory optimizations let you fine-tune models that would otherwise require multiple expensive GPUs on a single consumer GPU. A model that needs 48GB VRAM with standard training may need only 14GB with Unsloth. This means you can fine-tune on an RTX 4090 instead of renting an A100, saving significant compute costs.

How to use

1. Install:

curl -fsSL https://unsloth.ai/install.sh | sh

Or via pip:

pip install unsloth

1. Fine-tune a model:

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name='unsloth/Llama-3.2-3B-Instruct',
    max_seq_length=2048,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=16,
    target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj'],
)

# Train with your dataset using standard HuggingFace Trainer

1. Or use the web UI for no-code fine-tuning.

Example

Related on TokRepo

• Local LLM tools -- tools for running LLMs locally
• AI tools for coding -- developer tools for AI

Common pitfalls

• Unsloth optimizations are specific to certain model architectures. Check compatibility before starting a training run with a new model.
• 4-bit training (QLoRA) reduces VRAM usage further but may slightly affect model quality compared to full-precision LoRA.
• The web UI is convenient for getting started but the Python API provides more control for advanced training configurations.