Artillery — Modern Load Testing for HTTP, WebSocket & More

Introduction

Artillery is a Node.js-based load testing toolkit for HTTP, WebSocket, Socket.IO, gRPC, Kinesis, and custom protocols. It started as a simple CLI and has grown into a full distributed testing platform — the same YAML scenario runs on a laptop, in GitHub Actions, or scaled out across AWS Fargate with one flag.

What Artillery Does

• Executes YAML-defined scenarios against HTTP, WebSocket, Socket.IO, gRPC, Playwright, and more.
• Generates arrival-rate–based traffic (open model) so results reflect production load patterns.
• Exports metrics to Datadog, CloudWatch, Honeycomb, InfluxDB, Prometheus, and OpenTelemetry.
• Runs browser tests with Playwright engine for real end-user journey load testing.
• Distributes load across AWS Fargate or Kubernetes with a single artillery run-fargate command.

Architecture Overview

Artillery Core is a Node.js runtime that interprets YAML scenarios into a set of virtual-user pipelines. Each phase produces a Poisson-style arrival schedule at the configured rate, and each virtual user walks through its flow — HTTP requests, variable capture/setting, think-time, assertions — independently. Engines for non-HTTP protocols plug in as npm modules. For scale-out, artillery run-fargate packages the scenario into a task image and launches N workers with result aggregation over SQS.

Self-Hosting & Configuration

• Install via npm or the Docker image artilleryio/artillery:latest for CI.
• Keep shared data in CSV files referenced via the payload option for parameterized runs.
• Use hooks in processor JS files to sign requests, generate tokens, or shape payloads.
• Configure thresholds (ensure.thresholds) so CI fails on p95 / error-rate regressions.
• Ship metrics via plugins.publish-metrics — works out of the box with OpenTelemetry OTLP.

Key Features

• Scenario-level DSL with variables, conditionals, loops, and request chaining.
• Plugin ecosystem: metrics, APDEX, expect-style assertions, custom engines.
• Playwright engine turns end-to-end user journeys into performance tests.
• Distributed runs on AWS Fargate without provisioning infrastructure.
• OpenAPI / Swagger importer scaffolds a scenario from a spec in seconds.

Comparison with Similar Tools

• k6 — JS-scripted, Go engine; Artillery is YAML-first with a JS extension surface.
• JMeter — mature and GUI-driven; Artillery is code-first and CI-native.
• Gatling — Scala-based, high performance; steeper learning curve.
• Locust — Python-based user classes; Artillery focuses on arrival-rate models.
• Vegeta — laser-focused on constant HTTP rate; Artillery covers more protocols and scenarios.

FAQ

Q: Is Artillery free? A: The CLI and core engine are open-source. Artillery Cloud is an optional SaaS for dashboards.

Q: Can I run distributed tests without AWS? A: Yes — the k8s-based runner is community-supported, or run Artillery processes yourself and aggregate metrics in Prometheus.

Q: How does it handle authentication? A: Use processor hooks to attach tokens, or the built-in beforeRequest handlers.

Q: Can it replay production traffic? A: With the playback plugin and HAR imports; otherwise scenarios are usually richer than raw replay.

Sources

• https://github.com/artilleryio/artillery
• https://www.artillery.io/docs