# whichllm — Find the Best Local LLM for Your Hardware

> A CLI tool that benchmarks and ranks local language models based on your actual hardware capabilities, helping you pick the fastest model that fits your VRAM.

## Install

Save as a script file and run:

# whichllm — Find the Best Local LLM for Your Hardware

## Quick Use
```bash
pip install whichllm
whichllm scan
# Lists recommended models ranked by performance on your GPU
whichllm bench --model mistral-7b-q4
```

## Introduction
whichllm is a command-line tool that identifies the best-performing local LLM for your specific hardware. Instead of guessing which model fits your VRAM, it scans your system, checks available memory, and ranks models based on real benchmark data.

## What whichllm Does
- Detects GPU type, VRAM capacity, and system RAM automatically
- Ranks compatible models by inference speed on similar hardware
- Filters models by quantization level, parameter count, and task type
- Runs local benchmarks to measure actual tokens-per-second throughput
- Recommends the highest-quality model that fits your available memory

## Architecture Overview
whichllm queries your system for GPU and memory information, then cross-references a curated database of model benchmarks collected from community hardware profiles. It calculates which models fit in your available VRAM at various quantization levels and ranks them by quality-adjusted throughput. The optional bench command runs inference locally to measure real performance.

## Self-Hosting & Configuration
- Install via pip: `pip install whichllm`
- No configuration file required; works out of the box
- Supports NVIDIA, AMD, and Apple Silicon GPUs
- Benchmark data is fetched from the project's community database
- Offline mode available with cached benchmark data

## Key Features
- Hardware-aware model recommendations instead of trial-and-error
- VRAM-fit calculation accounting for quantization and context length
- Community-sourced benchmark data across hundreds of GPU configurations
- Side-by-side comparison of multiple models on your hardware profile
- Integration with Ollama and llama.cpp for direct model downloads

## Comparison with Similar Tools
- **Ollama** — serves models but does not recommend which one fits; whichllm fills the selection gap
- **LM Studio** — GUI-based model browser; whichllm is CLI-first with hardware-aware ranking
- **GPT4All** — bundles selected models; whichllm covers a broader model catalog with benchmarks
- **llama.cpp** — inference engine; whichllm helps choose which model to run on it

## FAQ
**Q: Does it download models automatically?**
A: No. whichllm recommends models and provides download commands. You choose which to install.

**Q: Which GPUs are supported?**
A: NVIDIA GPUs via CUDA, AMD GPUs via ROCm, and Apple Silicon via Metal are all detected.

**Q: How accurate are the benchmark rankings?**
A: Rankings are based on community-reported data from matching hardware. Your actual results may vary slightly.

**Q: Can I contribute my own benchmarks?**
A: Yes. Running `whichllm bench` generates results you can submit to the community database.

## Sources
- https://github.com/Andyyyy64/whichllm
- https://pypi.org/project/whichllm/

---
Source: https://tokrepo.com/en/workflows/asset-957b85f9
Author: Script Depot