pip-tools — Python Dependency Pinning and Compilation

Introduction

pip-tools bridges the gap between loosely specified Python dependencies and fully reproducible installs. It compiles abstract requirements into concrete, pinned versions and then synchronizes a virtual environment to match, ensuring consistent builds across development, CI, and production.

What pip-tools Does

• Resolves dependency trees from high-level requirement specifications into pinned versions
• Generates requirements.txt files with exact versions and hashes for every transitive dependency
• Synchronizes a virtual environment to match the lock file, removing packages not listed
• Supports multiple output files for separating dev, test, and production dependencies
• Integrates with existing pip workflows without requiring a new project format

Architecture Overview

pip-tools consists of two commands. pip-compile reads .in files containing abstract dependency specifications, resolves the full transitive dependency tree using pip's resolver, and writes a .txt file with pinned versions. pip-sync reads the pinned file and adjusts the active virtual environment to match exactly, installing missing packages and uninstalling extras. The resolution process respects constraints files, extras, and platform markers, producing deterministic output suitable for locking.

Self-Hosting & Configuration

• Install pip-tools into your virtual environment or globally via pipx
• Create requirements.in files listing direct dependencies with version ranges
• Run pip-compile to generate the pinned requirements.txt lock file
• Use pip-sync to install exactly the pinned versions, removing unlisted packages
• Add --generate-hashes to pip-compile for supply-chain verification via hash checking

Key Features

• Deterministic lock file generation from abstract dependency specifications
• Hash-based verification support for supply-chain security
• Layered requirements with constraint files for separating environments
• Compatible with pip's native resolver and existing requirements.txt format
• Works alongside any virtual environment tool (venv, virtualenv, conda)

Comparison with Similar Tools

• Poetry — Full project manager with its own pyproject.toml lock format; heavier but more integrated
• PDM — PEP 621-compliant manager with its own lock file; good for modern Python packaging
• uv — Rust-based pip replacement with fast resolution; overlapping compile/sync features
• pip freeze — Captures installed state but does not resolve from abstract specs or manage sync
• Pipenv — Combines pip and virtualenv with a Pipfile.lock; slower resolution, less flexible layering

FAQ

Q: Can pip-tools work with pyproject.toml? A: Yes. pip-compile can read dependencies from pyproject.toml and write the pinned output to requirements.txt.

Q: How do I manage dev vs production dependencies? A: Create separate .in files (requirements.in, dev-requirements.in) and compile each to its own .txt file. Use constraint references to keep versions aligned.

Q: Does pip-sync remove packages not in the requirements file? A: Yes. pip-sync uninstalls any package in the environment that is not listed in the pinned requirements file, except pip, setuptools, and pip-tools itself.

Q: How do I upgrade a single dependency? A: Run pip-compile --upgrade-package=flask to re-resolve only that package while keeping others pinned.

Sources

• https://github.com/jazzband/pip-tools
• https://pip-tools.readthedocs.io/