Agenda — Lightweight Job Scheduling for Node.js

Introduction

Agenda is a job scheduling library for Node.js that uses MongoDB as its backing store. It lets you define named jobs, schedule them with cron expressions or intervals, and run them with configurable concurrency. Because jobs persist in MongoDB, they survive process restarts and can be distributed across multiple worker instances.

What Agenda Does

• Schedules jobs using cron expressions, human-readable intervals, or specific dates
• Persists job definitions and state in MongoDB for crash recovery and distributed execution
• Supports job priorities, so higher-priority work runs before lower-priority tasks
• Controls concurrency with per-job and global limits to prevent resource exhaustion
• Provides lifecycle events (start, complete, fail) for monitoring and logging

Architecture Overview

Agenda stores job documents in a MongoDB collection. A polling loop queries for jobs whose nextRunAt is in the past, locks them with an atomic update, and dispatches them to handler functions defined in your application code. Multiple Agenda instances can share the same MongoDB collection, effectively forming a distributed worker pool. Job locking uses MongoDB's findOneAndUpdate to prevent double-execution, and a configurable lock lifetime handles crashed workers.

Self-Hosting & Configuration

• Requires a running MongoDB instance (local, Atlas, or self-hosted)
• Configure the connection string via new Agenda({ db: { address: 'mongodb://...' } })
• Set processEvery to control how frequently Agenda polls for new jobs (default: 5 seconds)
• Adjust maxConcurrency and defaultConcurrency to limit parallel job execution
• Use agenda.on('fail', callback) to hook into failures for alerting and retry logic

Key Features

• MongoDB-backed persistence means jobs survive restarts and deploys without losing state
• Human-readable scheduling API: agenda.every('1 hour', 'jobName') or agenda.schedule('tomorrow at noon', 'jobName')
• Built-in job locking prevents the same job from running simultaneously on multiple workers
• Supports job-specific data payloads passed at schedule time and available in the handler
• Graceful shutdown with agenda.stop() waits for running jobs to finish before exiting

Comparison with Similar Tools

• BullMQ — uses Redis instead of MongoDB; better suited if you already run Redis and need advanced queue patterns
• node-cron — in-memory only with no persistence; Agenda survives process crashes
• Bree — runs jobs in worker threads without external storage; Agenda provides persistence and multi-instance coordination
• Celery — Python-based; Agenda is the Node.js equivalent with a simpler setup
• pg-boss — uses PostgreSQL instead of MongoDB; choose based on your existing database stack

FAQ

Q: Can I run multiple Agenda instances for high availability? A: Yes. Multiple instances sharing the same MongoDB collection form a worker pool with built-in job locking to prevent duplicate execution.

Q: How do I retry failed jobs? A: Listen to the fail event and call job.schedule('in 5 minutes').save() to reschedule, or set job.attrs.failCount limits in your handler.

Q: Does Agenda support delayed or one-time jobs? A: Yes. Use agenda.schedule('in 20 minutes', 'jobName', data) for delayed one-time execution.

Q: What happens if MongoDB goes down while jobs are running? A: Running jobs continue to completion, but new jobs cannot be picked up or saved until the connection is restored. Agenda reconnects automatically.

Sources

• https://github.com/agenda/agenda
• https://github.com/agenda/agenda#readme