detect-secrets — Enterprise Secret Scanning for Codebases

Introduction

detect-secrets is an enterprise-friendly tool developed by Yelp for finding and preventing secrets (API keys, passwords, tokens) from being committed to source code. Unlike regex-only scanners, it uses a plugin architecture with heuristics that significantly reduce false positives.

What detect-secrets Does

• Scans files for high-entropy strings, known secret patterns, and keyword matches
• Maintains a baseline file to track known secrets and allowlisted entries
• Provides an interactive audit mode for triaging detected secrets
• Integrates as a pre-commit hook to block secrets before they reach the repository
• Supports custom plugin development for organization-specific secret patterns

Architecture Overview

detect-secrets uses a plugin-based architecture where each detection method (high entropy, regex patterns, keyword matching) is an independent plugin. The scan engine iterates through files line-by-line, running each plugin and collecting potential secrets. Results are stored in a JSON baseline file that supports allowlisting and incremental scanning. The tool is designed to be deterministic across runs.

Self-Hosting & Configuration

• Install via pip in any Python 3.8+ environment
• Generate a baseline with detect-secrets scan at the repository root
• Add .secrets.baseline to version control for team-wide tracking
• Configure plugins and sensitivity in a .detect-secrets.yaml or via CLI flags
• Integrate with pre-commit framework by adding the hook to .pre-commit-config.yaml

Key Features

• Low false-positive rate through multi-signal detection (entropy + patterns + context)
• Baseline-driven workflow that tracks known secrets across the team
• Interactive audit mode for efficient human review
• Plugin architecture supporting custom detectors
• CI/CD integration via pre-commit hooks or direct CLI invocation

Comparison with Similar Tools

• Gitleaks — Faster scanning with regex rules, but higher false-positive rate; detect-secrets has better heuristics
• TruffleHog — Searches git history deeply; detect-secrets focuses on current state with baseline tracking
• git-secrets — AWS-focused and simpler; detect-secrets is more general-purpose and extensible
• Talisman — Go-based pre-push hook; detect-secrets offers richer audit and baseline workflows
• GitHub Secret Scanning — Cloud-only and partner-pattern-focused; detect-secrets runs anywhere and is customizable

FAQ

Q: How is detect-secrets different from Gitleaks? A: detect-secrets uses a baseline-and-audit workflow with lower false positives. Gitleaks is faster for one-shot scans but produces more noise.

Q: Can it scan git history? A: By default it scans the working tree. Use --all-files with git log piping for history scanning, though TruffleHog is better suited for deep history analysis.

Q: How do I allowlist a known false positive? A: Run detect-secrets audit .secrets.baseline and mark entries as false positives. They will be excluded from future scans.

Q: Does it support custom secret patterns? A: Yes. Write a Python plugin implementing the BasePlugin interface and register it in your configuration.

Sources

• https://github.com/Yelp/detect-secrets
• https://github.com/Yelp/detect-secrets#quickstart