Kornia — Differentiable Computer Vision Library for PyTorch

Introduction

Kornia is an open-source computer vision library that reimplements classical vision algorithms as differentiable PyTorch operators. This means every operation, from Gaussian blurring to perspective transforms, can be included in a neural network training loop and optimized via backpropagation, bridging the gap between traditional CV and deep learning.

What Kornia Does

• Provides differentiable geometric transforms: rotation, scaling, affine, homography, and camera models
• Implements image filtering: Gaussian blur, Sobel, Laplacian, median, bilateral
• Offers color space conversions: RGB, HSV, LAB, grayscale, YUV, and more
• Includes feature detection and matching: LoFTR, DISK, KeyNet, and classical detectors
• Supplies data augmentation pipelines that run on GPU with autograd support

Architecture Overview

Kornia mirrors the OpenCV API structure but every function operates on PyTorch tensors and supports batched inputs on GPU. Internally, operations are implemented using pure PyTorch ops and custom CUDA kernels where needed. The augmentation module uses a container system similar to torchvision transforms but runs entirely on GPU and preserves gradients. Kornia AI adds pre-trained models for detection, segmentation, and depth estimation.

Self-Hosting & Configuration

• Install with pip install kornia or pip install kornia[x] for extra features
• Requires PyTorch 1.9+ and works on CPU or CUDA-enabled GPUs
• All operations accept batched tensors in BCHW format (batch, channels, height, width)
• Augmentation pipelines are configured as composable modules with per-transform probability
• Integrates directly into PyTorch training loops and Lightning modules

Key Features

• Full autograd support enables learning geometric and photometric parameters
• GPU-accelerated batch processing for high-throughput image pipelines
• OpenCV-compatible API reduces the learning curve for CV practitioners
• Augmentation module replaces CPU-bound torchvision transforms with GPU operations
• Active maintenance with regular releases tracking PyTorch versions

Comparison with Similar Tools

• OpenCV — the standard CV library; not differentiable and primarily CPU-based
• torchvision — PyTorch vision utilities; fewer geometric operations, limited differentiability
• Albumentations — fast augmentation library; CPU-only, not differentiable
• Pillow/PIL — basic image processing; single-image, CPU, not differentiable
• scikit-image — NumPy-based algorithms; no GPU or autograd support

FAQ

Q: Can I use Kornia without a GPU? A: Yes. All operations work on CPU tensors. GPU is recommended for batch processing speed.

Q: Does Kornia replace OpenCV? A: For differentiable pipelines and GPU processing, yes. For video I/O, codec handling, and UI functions, OpenCV is still needed.

Q: Can I backpropagate through geometric transforms? A: Yes. Operations like rotation angle, scale factor, and homography matrices support gradients, enabling them to be learned parameters.

Q: How does augmentation performance compare to CPU libraries? A: GPU augmentations in Kornia can be an order of magnitude faster than CPU alternatives when processing large batches, and they avoid CPU-GPU data transfer bottlenecks.

Sources

• https://github.com/kornia/kornia
• https://kornia.readthedocs.io/