BullMQ — Premium Message Queue for Node.js and Redis

Introduction

BullMQ is the successor to Bull, the most popular Redis-based queue for Node.js. It provides a battle-tested foundation for background job processing with features like delayed execution, rate limiting, job prioritization, and parent-child job flows, all backed by atomic Redis operations.

What BullMQ Does

• Enqueues jobs into Redis and processes them in worker processes with configurable concurrency
• Supports delayed jobs, cron-style repeatable jobs, and job prioritization
• Implements parent-child job flows where a parent job waits for all children to complete
• Provides rate limiting per queue to control throughput against external API limits
• Emits lifecycle events (completed, failed, stalled) for monitoring and integration

Architecture Overview

BullMQ uses Redis Streams and sorted sets for reliable message delivery. Producers add jobs to a queue, which are stored as Redis entries. Workers claim jobs using the XREADGROUP command, ensuring each job is delivered to exactly one worker. All state transitions are performed via Lua scripts executed atomically inside Redis. Stalled job detection runs periodically to reclaim jobs from crashed workers. The library is written in TypeScript with full type definitions.

Self-Hosting & Configuration

• Requires Redis 5.0+ (Redis 7+ recommended for optimal performance)
• Install via npm; configure the Redis connection via IORedis options passed to Queue and Worker constructors
• Set worker concurrency with the concurrency option to control parallel job processing
• Use QueueScheduler (or built-in scheduler in v4+) for delayed and repeatable job support
• Monitor queues with the companion Bull Board or Arena web UI dashboards

Key Features

• Atomic Redis operations via Lua scripts guarantee exactly-once job delivery
• Job flows enable parent-child dependencies for complex multi-step workflows
• Rate limiter controls how many jobs per time window a worker processes
• Sandboxed processors run job handlers in separate Node.js processes for isolation
• Full TypeScript support with strong typing for job data and return values

Comparison with Similar Tools

• Celery — Python-based distributed task queue; BullMQ is the Node.js equivalent
• Asynq — Go-based Redis task queue; similar Redis-backed design for the Go ecosystem
• Sidekiq — Ruby background job processor on Redis; mature and feature-rich for Ruby apps
• Bee-Queue — simpler Node.js Redis queue; fewer features than BullMQ
• RabbitMQ — full message broker with protocol support; heavier than a Redis-backed library

FAQ

Q: What is the difference between Bull and BullMQ? A: BullMQ is a complete rewrite of Bull with a cleaner API, TypeScript-first design, job flows, and improved performance. New projects should use BullMQ.

Q: Can BullMQ handle millions of jobs? A: Yes. Redis handles the storage, and BullMQ processes jobs efficiently. For very high throughput, scale horizontally by adding more worker instances.

Q: Does BullMQ support job persistence? A: Jobs are stored in Redis. As long as Redis data is persisted (RDB or AOF), jobs survive restarts.

Q: How do I monitor BullMQ queues? A: Use Bull Board (a web dashboard) or Taskforce.sh (a hosted monitoring service) to visualize queues, jobs, and worker status.

Sources

• https://github.com/taskforcesh/bullmq
• https://docs.bullmq.io/