Weights & Biases — ML Experiment Tracking and Visualization Platform

Introduction

Weights & Biases (W&B) is an open-source ML experiment tracking library that logs metrics, hyperparameters, model checkpoints, and datasets during training. The wandb Python client is open source and can report to either the hosted W&B service or a self-hosted server.

What Weights & Biases Does

• Logs training metrics, system metrics, and hyperparameters automatically during model training
• Generates interactive dashboards for comparing runs across experiments
• Tracks datasets and model artifacts with versioning and lineage
• Supports hyperparameter sweeps with built-in Bayesian and grid search strategies
• Integrates natively with PyTorch, TensorFlow, Keras, Hugging Face Transformers, and Lightning

Architecture Overview

The W&B client library instruments training loops by intercepting metric calls and streaming data to a backend. Locally, it writes run data to a wandb/ directory. The data syncs to a W&B server (cloud or self-hosted) where it is stored in a structured format. The web UI queries this data to render charts, tables, and comparison views. Artifacts (models, datasets) are content-addressed and stored with dependency graphs for reproducibility.

Self-Hosting & Configuration

• Install the client with pip install wandb on Python 3.7+
• Authenticate with wandb login using an API key from your W&B account
• Self-host the W&B server via Docker or Kubernetes using the official Helm chart
• Set WANDB_BASE_URL to point at a self-hosted instance instead of the cloud service
• Configure offline mode with WANDB_MODE=offline for air-gapped environments, then sync later

Key Features

• Automatic logging hooks for major ML frameworks reduce instrumentation effort
• Interactive run comparison with parallel coordinates, scatter plots, and custom panels
• Artifact versioning tracks datasets and models with lineage and dependency graphs
• Sweep agent runs hyperparameter searches with configurable strategies
• Reports combine charts, markdown, and code into shareable documents

Comparison with Similar Tools

• MLflow — Open-source experiment tracking with broader model registry; W&B provides richer visualization
• TensorBoard — Free visualization for TensorFlow; W&B adds cross-framework support and collaboration
• Neptune — Hosted experiment tracker; W&B's open-source client allows self-hosting
• ClearML — End-to-end MLOps; W&B focuses on experiment tracking and visualization
• SwanLab — Lightweight alternative with similar tracking features and a modern UI

FAQ

Q: Is Weights & Biases free to use? A: The client library is open source (Apache-2.0). The hosted service has a free tier for individuals and paid plans for teams. Self-hosting requires a license.

Q: Can I use W&B without an internet connection? A: Yes. Set WANDB_MODE=offline to log locally, then sync with wandb sync when connectivity is restored.

Q: Which ML frameworks does it support? A: PyTorch, TensorFlow, Keras, Hugging Face Transformers, Lightning, JAX, scikit-learn, XGBoost, and more via automatic integrations.

Q: How does it compare to MLflow? A: Both track experiments and models. W&B offers more interactive visualization and collaboration features; MLflow provides a broader open-source model registry and deployment workflow.

Sources

• https://github.com/wandb/wandb
• https://docs.wandb.ai/