Chaos Mesh — Cloud-Native Chaos Engineering for Kubernetes

What it is

Chaos Mesh is a CNCF incubating project that provides a chaos engineering platform for Kubernetes. It lets platform and SRE teams run controlled, reproducible failure experiments against live clusters by expressing faults as Kubernetes Custom Resource Definitions (CRDs). This means experiments are versioned, scheduled, and gated in CI exactly like any other Kubernetes resource.

Chaos Mesh is suited for DevOps engineers, SREs, and platform teams who need to validate resilience claims before incidents happen in production.

How it saves time or tokens

Without Chaos Mesh, teams write ad-hoc bash scripts or manually kill pods to test resilience. Chaos Mesh replaces that fragile approach with declarative CRDs that can be applied, reverted, and automated in CI pipelines. A network-latency experiment that would take an hour to set up manually can be defined in a single YAML and applied in seconds. The built-in Dashboard provides a visual workflow editor that further reduces setup time for complex multi-step game-day scenarios.

How to use

1. Install Chaos Mesh via Helm into your cluster:

helm repo add chaos-mesh https://charts.chaos-mesh.org
helm install chaos-mesh chaos-mesh/chaos-mesh \
  --namespace=chaos-mesh --create-namespace --version 2.6.3

1. Define a chaos experiment as a CRD. For example, inject 500ms network latency into a service:

apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
metadata:
  name: web-latency
  namespace: default
spec:
  action: delay
  mode: all
  selector:
    labelSelectors:
      app: web
  delay:
    latency: '500ms'
    jitter: '50ms'
  duration: '2m'

1. Apply the experiment with kubectl apply -f latency.yaml and observe your service behavior in the Chaos Mesh Dashboard or your existing monitoring stack.

Example

A multi-step workflow that kills a database pod, then injects network partition, then verifies recovery:

apiVersion: chaos-mesh.org/v1alpha1
kind: Workflow
metadata:
  name: db-resilience-test
spec:
  entry: serial-steps
  templates:
    - name: serial-steps
      templateType: Serial
      children:
        - kill-db
        - network-partition
    - name: kill-db
      templateType: PodChaos
      podChaos:
        action: pod-kill
        mode: one
        selector:
          labelSelectors:
            app: postgres
    - name: network-partition
      templateType: NetworkChaos
      networkChaos:
        action: partition
        mode: all
        selector:
          labelSelectors:
            app: web
        direction: both
        duration: '60s'

Related on TokRepo

• DevOps automation tools — Browse more infrastructure and operations tools
• Self-hosted tools — Explore self-hosted software for your infrastructure

Common pitfalls

• Running chaos experiments in production without proper blast-radius controls. Always use label selectors and namespace scoping to limit the impact.
• Forgetting to set a duration field, which can leave faults running indefinitely and cause real outages.
• Not integrating experiments into CI/CD. One-off manual chaos runs provide limited value compared to automated regression chaos tests.