# Dramatiq — Fast Distributed Task Queue for Python

> A reliable, high-performance background task processing library for Python with support for RabbitMQ and Redis brokers, automatic retries, rate limiting, and built-in result storage.

## Install

Save in your project root:

# Dramatiq — Fast Distributed Task Queue for Python

## Quick Use
```bash
# Install with RabbitMQ broker
pip install dramatiq[rabbitmq]

# Or with Redis broker
pip install dramatiq[redis]

# Define a task (tasks.py)
cat > tasks.py << 'PYEOF'
import dramatiq

@dramatiq.actor
def send_email(to, subject):
    print(f"Sending email to {to}: {subject}")
PYEOF

# Start a worker
dramatiq tasks

# Enqueue a task (from another script or shell)
python3 -c "from tasks import send_email; send_email.send('user@example.com', 'Hello')"
```

## Introduction
Dramatiq is a distributed task processing library for Python designed as a modern alternative to Celery. It focuses on simplicity, reliability, and performance. Tasks are defined as decorated functions and dispatched to a message broker (RabbitMQ or Redis), where worker processes pick them up and execute them. Dramatiq handles retries, dead-letter queues, and rate limiting out of the box with sensible defaults.

## What Dramatiq Does
- Processes background tasks asynchronously using RabbitMQ or Redis as a message broker
- Retries failed tasks automatically with exponential backoff and configurable limits
- Groups related tasks with pipelines and groups for fan-out/fan-in workflows
- Enforces rate limits and time limits per actor to prevent resource exhaustion
- Stores task results in Redis or Memcached for retrieval by the calling code

## Architecture Overview
Dramatiq uses a multi-process, multi-thread worker architecture. Each worker process spawns a configurable number of threads that consume messages from broker queues. Messages are acknowledged only after successful processing, ensuring at-least-once delivery. The middleware pipeline wraps each message processing step, providing hooks for retries, time limits, rate limiting, and logging. Actors (task definitions) are registered at import time, and workers discover them by importing the specified modules. The broker abstraction supports both RabbitMQ (via pika) and Redis (via redis-py) with consistent semantics.

## Self-Hosting & Configuration
- Install a broker: RabbitMQ (`sudo apt install rabbitmq-server`) or Redis (`sudo apt install redis-server`)
- Set the broker URL via environment variable: `DRAMATIQ_BROKER_URL=amqp://localhost`
- Start workers with `dramatiq module_name` pointing to the Python module containing your actors
- Adjust worker concurrency with `--processes` and `--threads` flags
- Add the results middleware (`dramatiq.results`) if you need to retrieve return values from tasks

## Key Features
- Sensible defaults: retries with backoff, time limits, and graceful shutdown work out of the box
- Middleware architecture lets you add custom pre/post-processing logic to every task
- Pipelines chain tasks sequentially and groups execute tasks in parallel with barrier synchronization
- Priority queues let urgent tasks skip ahead of lower-priority work
- Prometheus metrics integration for monitoring queue depths, processing times, and error rates

## Comparison with Similar Tools
- **Celery** — the established Python task queue; Dramatiq offers a simpler API, fewer configuration options, and better default behavior
- **RQ (Redis Queue)** — simpler but single-threaded per worker; Dramatiq uses threads for higher throughput
- **Huey** — lightweight with SQLite support; Dramatiq targets production workloads with RabbitMQ/Redis
- **Taskiq** — async-first task queue; Dramatiq is synchronous by default but can run async actors
- **BullMQ** — Node.js task queue; Dramatiq fills the same role in the Python ecosystem

## FAQ
**Q: Why choose Dramatiq over Celery?**
A: Dramatiq has a simpler API, fewer footguns (like prefetching issues), sane defaults for retries and time limits, and better documentation per line of code.

**Q: Does Dramatiq support periodic/scheduled tasks?**
A: Not directly. Use the companion library `APScheduler` or `dramatiq-crontab` to schedule recurring tasks that enqueue Dramatiq actors.

**Q: Can I use Dramatiq with Django?**
A: Yes. The `django-dramatiq` package integrates Dramatiq with Django's settings and management commands.

**Q: How does Dramatiq handle task failures?**
A: Failed tasks are retried with exponential backoff up to a configurable limit. After exhausting retries, they are moved to a dead-letter queue for manual inspection.

## Sources
- https://github.com/Bogdanp/dramatiq
- https://dramatiq.io/

---
Source: https://tokrepo.com/en/workflows/asset-27a84a61
Author: AI Open Source