# Beanstalkd — Simple Fast Work Queue

> A lightweight, high-performance in-memory work queue with a simple protocol, designed for asynchronous job processing with O(log n) operations.

## Install

Save as a script file and run:

# Beanstalkd — Simple Fast Work Queue

## Quick Use
```bash
# Install and start
brew install beanstalkd   # or apt-get install beanstalkd
beanstalkd -l 127.0.0.1 -p 11300 &

# Put a job (using netcat)
echo -e "put 0 0 60 5
hello
" | nc localhost 11300
```

## Introduction
Beanstalkd is a simple, fast, general-purpose work queue originally designed to reduce page latency in high-volume web applications by running time-consuming tasks asynchronously. Written in C, it provides a lightweight protocol over TCP with tubes (named queues), priority ordering, and delayed jobs.

## What Beanstalkd Does
- Provides named tubes for logically separating job queues across different task types
- Supports job priorities so urgent work is processed before background tasks
- Offers delayed jobs that become available after a specified number of seconds
- Implements time-to-run (TTR) with automatic job release if a worker dies mid-processing
- Persists jobs to a binlog on disk for crash recovery

## Architecture Overview
Beanstalkd is a single-threaded, event-driven server written in C using an internal event loop. Jobs are stored in an in-memory priority queue backed by a min-heap, giving O(log n) put and reserve operations. Each tube maintains its own ready, delayed, and buried queues. The text-based protocol runs over TCP, making it easy to implement clients in any language. Optional binlog persistence writes jobs to an append-only file for durability.

## Self-Hosting & Configuration
- Single binary with zero dependencies; compile from source or install via package manager
- Configure listen address and port with `-l` and `-p` flags
- Enable persistence with `-b /path/to/binlog` for crash recovery
- Set max job size with `-z` (default 65535 bytes)
- Monitor with `stats` and `stats-tube` commands over the protocol

## Key Features
- Sub-millisecond latency for put and reserve operations
- Minimal memory footprint suitable for embedded and resource-constrained environments
- Client libraries available in 20+ languages including Python, Ruby, PHP, Go, and Java
- Buried job state for manual inspection of failed jobs without losing them
- Kick command to move buried or delayed jobs back to the ready queue

## Comparison with Similar Tools
- **Redis (with lists)** — More versatile but heavier; Beanstalkd is purpose-built for job queues with priorities and TTR
- **RabbitMQ** — Full AMQP broker with clustering and routing; Beanstalkd is simpler with lower overhead
- **Amazon SQS** — Managed cloud service; Beanstalkd is self-hosted with no external dependencies
- **Celery** — Python task framework that can use Beanstalkd as a broker; Beanstalkd is the transport layer

## FAQ
**Q: Does Beanstalkd support clustering or replication?**
A: No, Beanstalkd is a single-server daemon. For high availability, run multiple instances behind application-level routing.

**Q: What happens if Beanstalkd crashes?**
A: With binlog enabled, jobs are recovered on restart. Without it, in-memory jobs are lost.

**Q: Is there a web UI for monitoring?**
A: Community tools like Beanstalk Console and Aurora provide web-based monitoring interfaces.

**Q: How does TTR work?**
A: When a worker reserves a job, it has TTR seconds to delete or release it. If TTR expires, the job is automatically released back to the ready queue for another worker.

## Sources
- https://github.com/beanstalkd/beanstalkd
- https://beanstalkd.github.io/

---
Source: https://tokrepo.com/en/workflows/asset-a691cd51
Author: Script Depot