# GPyTorch — Efficient Gaussian Processes in PyTorch

> A PyTorch library for scalable Gaussian process models with GPU acceleration, enabling Bayesian optimization, uncertainty quantification, and probabilistic regression.

## Install

Save as a script file and run:

# GPyTorch — Efficient Gaussian Processes in PyTorch

## Quick Use
```bash
pip install gpytorch
```
```python
import torch, gpytorch

class ExactGPModel(gpytorch.models.ExactGP):
    def __init__(self, train_x, train_y, likelihood):
        super().__init__(train_x, train_y, likelihood)
        self.mean_module = gpytorch.means.ConstantMean()
        self.covar_module = gpytorch.kernels.ScaleKernel(
            gpytorch.kernels.RBFKernel()
        )
    def forward(self, x):
        return gpytorch.distributions.MultivariateNormal(
            self.mean_module(x), self.covar_module(x)
        )

train_x = torch.linspace(0, 1, 100)
train_y = torch.sin(train_x * 6.28) + torch.randn(100) * 0.2
likelihood = gpytorch.likelihoods.GaussianLikelihood()
model = ExactGPModel(train_x, train_y, likelihood)
```

## Introduction
GPyTorch is a Gaussian process library built on PyTorch that makes GP models as fast and scalable as deep neural networks. It uses GPU-accelerated linear algebra and modern approximation methods to handle datasets with millions of points, far beyond what traditional GP implementations can manage.

## What GPyTorch Does
- Implements exact and approximate Gaussian process inference on GPU
- Provides a modular kernel library with automatic differentiation for hyperparameter learning
- Scales to datasets with millions of observations using KISS-GP and inducing point methods
- Integrates with BoTorch for Bayesian optimization and experiment design
- Supports multi-task, deep kernel, and variational Gaussian process models

## Architecture Overview
GPyTorch represents GP models as PyTorch modules with lazy tensor operations. Kernel matrices are never fully materialized; instead, matrix-vector multiplies are computed on-the-fly using structured kernel interpolation (SKI) or inducing point methods. The CG (conjugate gradient) solver computes log-determinants and solves linear systems without forming dense matrices. Automatic differentiation through PyTorch handles hyperparameter optimization via type-II maximum likelihood.

## Self-Hosting & Configuration
- Install via pip alongside PyTorch with GPU support
- Define custom GP models by subclassing gpytorch.models.ExactGP or ApproximateGP
- Choose kernels from the library or compose custom kernels with arithmetic operations
- Control CG solver tolerance and maximum iterations for speed-accuracy tradeoffs
- Use gpytorch.settings context managers to configure computation precision

## Key Features
- GPU-accelerated kernel computations make GPs competitive with neural network training speed
- Lazy tensor algebra avoids materializing large kernel matrices in memory
- Modular design allows mixing kernels, likelihoods, and inference strategies
- Pre-conditioning strategies accelerate conjugate gradient convergence
- Tight integration with BoTorch enables production Bayesian optimization

## Comparison with Similar Tools
- **scikit-learn GaussianProcessRegressor** — CPU-only with O(n^3) scaling; GPyTorch uses GPU and scales to millions of points
- **GPflow** — TensorFlow-based GP library; GPyTorch uses PyTorch and benefits from its autograd ecosystem
- **BoTorch** — Bayesian optimization library built on top of GPyTorch; GPyTorch provides the GP layer
- **Stan** — general probabilistic programming; GPyTorch specializes in GP models with GPU acceleration
- **Pyro** — deep probabilistic programming in PyTorch; GPyTorch focuses specifically on efficient GP inference

## FAQ
**Q: How many data points can GPyTorch handle?**
A: With approximate methods (KISS-GP, inducing points), GPyTorch scales to millions of data points on a single GPU.

**Q: Can I use it for Bayesian optimization?**
A: Yes. GPyTorch is the GP backend for BoTorch, Meta's Bayesian optimization library used in production hyperparameter tuning.

**Q: Does it support multi-output GPs?**
A: Yes. GPyTorch provides multi-task and multi-output GP models with shared or independent kernels.

**Q: How does it compare to exact GP implementations?**
A: GPyTorch supports both exact inference (for smaller datasets) and scalable approximate inference for large datasets.

## Sources
- https://github.com/cornellius-gp/gpytorch
- https://gpytorch.ai/

---
Source: https://tokrepo.com/en/workflows/asset-0f760aa5
Author: Script Depot