# PyTorch Lightning — Scale Deep Learning Models with Zero Boilerplate

> A lightweight PyTorch wrapper that decouples research code from engineering, enabling training on any hardware from a laptop GPU to a thousand-node cluster without code changes.

## Install

Save in your project root:

# PyTorch Lightning — Scale Deep Learning Models with Zero Boilerplate

## Quick Use
```bash
pip install lightning
```
```python
import lightning as L
import torch

class LitModel(L.LightningModule):
    def __init__(self):
        super().__init__()
        self.layer = torch.nn.Linear(28*28, 10)

    def training_step(self, batch, batch_idx):
        x, y = batch
        return torch.nn.functional.cross_entropy(self.layer(x.view(x.size(0), -1)), y)

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=1e-3)

trainer = L.Trainer(max_epochs=5, accelerator="auto")
trainer.fit(model, dataloader)
```

## Introduction
PyTorch Lightning removes the boilerplate from PyTorch training loops while giving you full control over the research code. It handles distributed training, mixed precision, checkpointing, logging, and hardware abstraction so you can focus on the model logic rather than engineering plumbing.

## What PyTorch Lightning Does
- Abstracts away training loop boilerplate while keeping pure PyTorch in the model definition
- Automatically handles multi-GPU, multi-node, TPU, and Apple Silicon training
- Provides built-in 16-bit and bfloat16 mixed-precision training with no code changes
- Integrates checkpointing, early stopping, and learning rate scheduling out of the box
- Supports 15+ logging backends including TensorBoard, W&B, MLflow, and Neptune

## Architecture Overview
Lightning separates concerns into two core classes: LightningModule (your model and training logic) and Trainer (the engineering). The Trainer orchestrates the training loop, handles accelerator selection via Strategy plugins, manages precision via Precision plugins, and coordinates callbacks for extensibility. This plugin system allows swapping backends without modifying model code.

## Self-Hosting & Configuration
- Install via pip or conda with optional extras for specific accelerator support
- Configure hardware with Trainer flags like accelerator, devices, and strategy
- Set up distributed training across nodes using environment variables or SLURM integration
- Use YAML configs via LightningCLI for reproducible experiment management
- Customize the training loop by overriding Trainer hooks without rewriting the loop

## Key Features
- Train on 1 to 10,000+ GPUs with zero code changes to your model
- Built-in support for DeepSpeed and FSDP for large model training
- Automatic gradient accumulation, clipping, and scaling
- Fault-tolerant training with automatic resume from last checkpoint
- Extensive callback system for custom behavior at any point in the training cycle

## Comparison with Similar Tools
- **Plain PyTorch** — full control but requires writing boilerplate for distributed training, logging, and checkpointing
- **Hugging Face Trainer** — focused on NLP and Transformers; Lightning is model-agnostic
- **Keras** — higher-level abstraction with less control; Lightning preserves raw PyTorch access
- **Ignite** — event-based training; Lightning uses a more structured module-based approach
- **Fabric (Lightning)** — minimal wrapper for those who want less structure than full Lightning

## FAQ
**Q: Does Lightning add overhead to training speed?**
A: No measurable overhead. Lightning compiles down to the same PyTorch operations and has been benchmarked at equivalent speeds.

**Q: Can I still use raw PyTorch code inside Lightning?**
A: Yes. The LightningModule is a standard nn.Module, and you write pure PyTorch in training_step and other hooks.

**Q: How does Lightning compare to Fabric?**
A: Fabric is a lighter alternative within the same project that gives you distributed training without the full Trainer structure.

**Q: Does it support custom training loops beyond standard supervised learning?**
A: Yes. You can override any hook, use manual optimization for GANs, or implement complex multi-optimizer schedules.

## Sources
- https://github.com/Lightning-AI/pytorch-lightning
- https://lightning.ai/docs/pytorch/stable/

---
Source: https://tokrepo.com/en/workflows/asset-2695b27f
Author: AI Open Source