# pgmq — Lightweight Message Queue Built on PostgreSQL

> A PostgreSQL extension that provides a message queue with visibility timeouts, exactly-once delivery semantics, and no external dependencies.

## Install

Save in your project root:

# pgmq — Lightweight Message Queue Built on PostgreSQL

## Quick Use
```sql
-- Install the extension
CREATE EXTENSION pgmq;
-- Create a queue
SELECT pgmq.create('my_queue');
-- Send a message
SELECT pgmq.send('my_queue', '{"task": "process_order", "order_id": 42}'::jsonb);
-- Read a message (30-second visibility timeout)
SELECT * FROM pgmq.read('my_queue', 30, 1);
-- Delete after processing
SELECT pgmq.delete('my_queue', msg_id);
```

## Introduction
pgmq is a lightweight message queue implemented as a PostgreSQL extension. It brings message queue semantics — send, read with visibility timeout, delete, and archive — directly into PostgreSQL without requiring a separate broker. Applications already using PostgreSQL can add job queues and task processing without additional infrastructure.

## What pgmq Does
- Provides send, read, delete, and archive operations on named queues
- Implements visibility timeouts so messages are hidden from other consumers while being processed
- Supports batch operations for sending and reading multiple messages at once
- Archives processed messages to a separate table for auditing
- Offers both SQL function calls and client libraries for Python, Rust, and other languages

## Architecture Overview
Each queue is backed by two PostgreSQL tables — one for active messages and one for archived messages. The `read` function uses `SELECT ... FOR UPDATE SKIP LOCKED` for safe concurrent access. Visibility timeouts are enforced by setting a future timestamp on read; messages become visible again if not deleted before timeout. All operations run within PostgreSQL transactions, leveraging existing WAL replication and backup.

## Self-Hosting & Configuration
- Install as a PostgreSQL extension via `CREATE EXTENSION pgmq` (available on Trunk, pgxn, or from source)
- Works with PostgreSQL 12 and above
- No separate daemon or broker process required
- Queues are created and managed with SQL function calls
- Inherits PostgreSQL's replication, backup, and high-availability configurations

## Key Features
- Visibility timeout prevents duplicate processing without external coordination
- `SKIP LOCKED` based reads for high-concurrency consumer patterns
- Message archival for audit trails and reprocessing
- Batch send and read operations for throughput optimization
- Partitioned queue tables available for high-volume scenarios

## Comparison with Similar Tools
- **RabbitMQ** — Full-featured broker with protocols like AMQP; pgmq is simpler with no extra infrastructure
- **Redis Streams** — In-memory with persistence options; pgmq uses PostgreSQL's durable storage natively
- **Amazon SQS** — Managed cloud queue; pgmq is self-hosted and avoids vendor lock-in
- **Celery** — Task framework needing a broker; pgmq can serve as that broker within PostgreSQL

## FAQ
**Q: Does pgmq guarantee exactly-once delivery?**
A: It provides at-least-once delivery via visibility timeouts. Exactly-once processing depends on consumers deleting messages after successful handling.

**Q: How does pgmq handle failed consumers?**
A: If a consumer crashes before deleting a message, the visibility timeout expires and the message becomes available for another consumer to read.

**Q: Can pgmq handle high throughput?**
A: It performs well for moderate workloads. For very high message rates, a dedicated broker may be more appropriate, but partitioned queues help scale within PostgreSQL.

## Sources
- https://github.com/tembo-io/pgmq
- https://tembo.io/pgmq

---
Source: https://tokrepo.com/en/workflows/asset-c6ed42f0
Author: AI Open Source