PowerJob — Distributed Job Scheduling and Computing Platform

Introduction

PowerJob is a distributed job scheduling middleware for Java that goes beyond simple cron triggers. It supports workflow DAGs, MapReduce-style distributed computing, and multiple execution modes, all managed through a clean web console.

What PowerJob Does

• Schedules tasks using cron, fixed-rate, fixed-delay, or API-triggered execution
• Runs jobs across a cluster with automatic worker discovery and failover
• Supports MapReduce and broadcast execution modes for parallel processing
• Provides a workflow engine for building multi-step DAG pipelines
• Includes a web console for job management, log viewing, and execution history

Architecture Overview

PowerJob consists of a server (powerjob-server) and workers (powerjob-worker). The server stores job metadata in a relational database, handles scheduling via a time-wheel algorithm, and dispatches tasks to workers over HTTP or Akka. Workers register with the server on startup and report heartbeats. For MapReduce jobs, the server coordinates task splitting and result aggregation across worker nodes.

Self-Hosting & Configuration

• Deploy powerjob-server as a Spring Boot application backed by MySQL or PostgreSQL
• Add the powerjob-worker-spring-boot-starter to your application
• Configure powerjob.worker.server-address to point to your server cluster
• Access the web console at the server's HTTP port for job creation and monitoring
• Server supports H2 for development and MySQL/PostgreSQL for production

Key Features

• Multiple execution modes: standalone, broadcast, MapReduce, and Map
• Workflow engine for chaining jobs into directed acyclic graphs
• Built-in log collection and real-time log streaming in the console
• Automatic failover with configurable retry policies per job
• Container deployment mode for running jobs in isolated environments

Comparison with Similar Tools

• XXL-JOB — simpler scheduler with executor groups; PowerJob adds MapReduce computing and workflow DAGs
• Quartz — in-process job scheduler; PowerJob provides distributed coordination and a management console
• Apache Airflow — Python-based DAG orchestrator; PowerJob is Java-native with lower operational overhead
• Elastic-Job — ShardingSphere ecosystem; PowerJob offers a more modern console and richer execution modes

FAQ

Q: How does PowerJob differ from XXL-JOB? A: PowerJob adds distributed computing (MapReduce/Map), workflow DAGs, and container-based execution that XXL-JOB does not provide.

Q: Can I run PowerJob server in high availability mode? A: Yes. Deploy multiple server instances pointed at the same database; they coordinate via database-level locking.

Q: What languages can workers be written in? A: The primary SDK is Java/Spring Boot. Community-maintained agents exist for Go and Python.

Q: Does it support dynamic sharding? A: Yes. MapReduce and Map execution modes automatically distribute sub-tasks across available workers.

Sources

• https://github.com/PowerJob/PowerJob
• https://www.yuque.com/powerjob/guidence