tinygrad — Minimalist Deep Learning Framework

Introduction

tinygrad is a deep learning framework designed to be simple, small, and understandable. Created by George Hotz, it aims to prove that a fully featured ML framework can exist in a fraction of the code of PyTorch or TensorFlow while still supporting real model training and inference across multiple hardware backends.

What tinygrad Does

• Provides a NumPy-like Tensor API with automatic differentiation for gradient computation
• Supports model training and inference for common architectures including transformers and CNNs
• Compiles operations to multiple backends: CPU, CUDA, Metal, OpenCL, and custom accelerators
• Includes a lazy evaluation engine that fuses operations for optimal kernel generation
• Ships with implementations of models like LLaMA, Stable Diffusion, and BERT for reference

Architecture Overview

tinygrad uses a lazy evaluation graph where tensor operations build a computation graph of abstract operations. A scheduler groups these operations into optimized kernels, which are then lowered through a linearizer into backend-specific code (CUDA, Metal, LLVM, etc.). The entire stack from tensor API to code generation fits in under 10,000 lines, making it one of the most readable ML compilers available.

Self-Hosting & Configuration

• Install from PyPI: pip install tinygrad for CPU, or clone the repo for development
• GPU support activates automatically when CUDA or Metal drivers are detected
• Set BACKEND=CUDA or BACKEND=METAL environment variables to select a specific backend
• Debug operations with DEBUG=2 for kernel-level tracing and DEBUG=4 for generated code
• Run the test suite with python -m pytest test/ to verify your setup

Key Features

• Entire codebase under 10,000 lines — easy to read, modify, and contribute to
• Lazy evaluation with automatic kernel fusion for competitive performance
• Multi-backend support including experimental AMD and Qualcomm accelerator targets
• JIT compilation that generates optimized code per backend
• Active development toward a custom AI accelerator chip designed around tinygrad's compiler

Comparison with Similar Tools

• PyTorch — industry standard with massive ecosystem but orders of magnitude more code and complexity
• JAX — functional approach with XLA compilation but steeper learning curve and Google dependency
• MLX — Apple-focused framework with good Metal support but limited to Apple Silicon
• Micrograd — educational autograd engine but lacks real hardware backends and model support
• ONNX Runtime — inference-only runtime without training support or a tensor API

FAQ

Q: Can tinygrad train real models competitively? A: Yes, tinygrad can train models like LLaMA and ResNet. Performance is approaching PyTorch on supported hardware, though the ecosystem of pre-built training recipes is smaller.

Q: What hardware does tinygrad support? A: CPU (via LLVM or Clang), NVIDIA GPUs (CUDA/NV), Apple GPUs (Metal), AMD GPUs (HSA/HIP), and OpenCL devices. Experimental support exists for custom accelerators.

Q: Is tinygrad suitable for production inference? A: It can serve models in production, but the ecosystem around deployment tooling is less mature than PyTorch or ONNX Runtime.

Q: How does the codebase stay so small? A: tinygrad uses aggressive abstraction at the compiler level, representing all operations as a small set of primitives that lower uniformly across backends.

Sources

• https://github.com/tinygrad/tinygrad
• https://tinygrad.org