Apache Pulsar — Cloud-Native Distributed Messaging and Streaming

What it is

Apache Pulsar is a cloud-native distributed messaging and streaming platform that combines the capabilities of traditional message queues (like RabbitMQ) with event streaming (like Kafka) in a single system. It provides multi-tenancy, geo-replication, and tiered storage as built-in features rather than add-ons.

Pulsar is designed for platform teams and backend engineers who need a unified messaging layer that scales from simple pub-sub to complex event streaming without deploying separate systems for each use case.

How it saves time or tokens

Pulsar's architecture separates compute (brokers) from storage (BookKeeper), which means you can scale throughput and storage independently. This eliminates the rebalancing pain common with broker-storage-coupled systems. Multi-tenancy is built in, so a single Pulsar cluster can serve multiple teams with namespace-level isolation, reducing operational overhead.

The unified messaging model means you do not need to maintain separate Kafka clusters for streaming and RabbitMQ for queuing. One Pulsar cluster handles both patterns with topic-level configuration.

How to use

1. Start Pulsar with Docker: docker run -d --name pulsar -p 6650:6650 -p 8080:8080 apachepulsar/pulsar:latest bin/pulsar standalone.
2. Produce a message: bin/pulsar-client produce my-topic --messages 'hello pulsar'.
3. Consume messages: bin/pulsar-client consume my-topic -s my-sub --num-messages 0.

Example

# Start Pulsar standalone in Docker
docker run -d --name pulsar \
  -p 6650:6650 -p 8080:8080 \
  apachepulsar/pulsar:latest bin/pulsar standalone

# Produce messages
bin/pulsar-client produce my-topic --messages 'hello pulsar'

# Consume messages
bin/pulsar-client consume my-topic -s my-subscription --num-messages 0

# Python client
pip install pulsar-client

import pulsar

client = pulsar.Client('pulsar://localhost:6650')
producer = client.create_producer('my-topic')
producer.send('hello from python'.encode())
client.close()

Related on TokRepo

• DevOps tools -- infrastructure and operations tooling
• Self-hosted solutions -- open-source self-hosted platforms

Common pitfalls

• Pulsar standalone mode is for development only; production deployments require a ZooKeeper cluster and BookKeeper ensemble, which adds operational complexity.
• The broker-storage separation is powerful but means more moving parts to monitor; invest in observability (Prometheus metrics are built in) from day one.
• Client library support varies by language; Java and Python clients are most mature, while Go and Node.js clients may lag in feature parity.