Hypothesis — Property-Based Testing for Python

Introduction

Hypothesis generates random test inputs based on strategies you define, then shrinks failing examples to minimal reproducible cases. It catches bugs that fixed example-based tests never reach by exploring the input space systematically.

What Hypothesis Does

• Generates diverse test inputs from composable strategy combinators
• Automatically shrinks failing inputs to the simplest reproduction case
• Persists and replays previously discovered failures across test runs
• Integrates with pytest, unittest, and other standard test runners
• Provides strategies for primitives, collections, dates, text, and custom data

Architecture Overview

Hypothesis maintains an internal database of test examples and a conjecture engine that explores the input space using coverage-guided fuzzing techniques. When a test fails, the shrinker reduces the example through a series of simplification passes until no smaller failing input can be found.

Self-Hosting & Configuration

• Install via pip; works with Python 3.8+
• Configure via settings profiles (max examples, deadline, database backend)
• Use --hypothesis-show-statistics with pytest for run diagnostics
• Store the example database in .hypothesis/ (add to .gitignore or commit for CI reproducibility)
• Suppress health checks selectively when generating expensive data

Key Features

• 40+ built-in strategies covering all Python types and common patterns
• Stateful testing for protocol and API sequence validation
• Deadline detection to catch performance regressions
• Explicit example injection for known edge cases alongside generated ones
• Extensible with custom strategies via @composite or flatmap

Comparison with Similar Tools

• pytest parametrize — fixed examples only; Hypothesis explores the space
• QuickCheck (Haskell) — the original property-based tester; Hypothesis is its Python successor
• fast-check (JS) — similar approach for JavaScript/TypeScript
• Schemathesis — builds on Hypothesis specifically for OpenAPI/GraphQL API testing

FAQ

Q: How many examples does Hypothesis run per test? A: By default 100, configurable via settings(max_examples=N). More examples increase confidence but slow tests.

Q: Does Hypothesis work with pytest? A: Yes. Decorate test functions with @given() and run them with pytest normally. No special plugins needed.

Q: How does shrinking work? A: When a test fails, Hypothesis tries progressively simpler inputs (smaller numbers, shorter lists, earlier characters) while still triggering the failure, reporting the minimal case.

Q: Can I test stateful systems like APIs? A: Yes. Use RuleBasedStateMachine to define state transitions and let Hypothesis find sequences of operations that break invariants.

Sources

• https://github.com/HypothesisWorks/hypothesis
• https://hypothesis.readthedocs.io