BuildKit — Concurrent, Cache-Efficient OCI Image Builder

Introduction

BuildKit was born inside the Moby project to replace the aging Docker 1.x build engine. Classic docker build was single-threaded, cached poorly, and could not build for multiple platforms or export to custom targets. BuildKit rewrote the pipeline around a content-addressable LLB (low-level build) graph that can be evaluated in parallel, cached across machines, and driven by pluggable frontends — so Dockerfiles are just one input language among many.

What BuildKit Does

• Executes builds as a DAG of cache-aware operations, running independent stages in parallel.
• Supports remote cache import/export to registries, S3, GHA, and local tarballs.
• Builds for multiple architectures in one invocation via QEMU or native worker pools.
• Runs rootless, inside unprivileged containers, or as a Kubernetes daemonset.
• Exposes pluggable frontends so you can build with Dockerfile, BuildPacks, Earthly, or custom DSLs.

Architecture Overview

BuildKit splits into a long-running daemon (buildkitd) and stateless clients (buildctl, docker buildx). The client ships an LLB graph to the daemon, which resolves and caches each vertex in a content-addressable store and executes operations in containerd or runc workers. Frontends like dockerfile.v0 run as pluggable gateway images that translate source languages into LLB — meaning Dockerfile features can be updated just by pointing at a newer frontend image, no daemon upgrade needed. Outputs are pluggable too: OCI tarball, image push, local filesystem, or docker load.

Self-Hosting & Configuration

• Run buildkitd as a Docker container, systemd unit, or Kubernetes Deployment.
• Configure storage via /etc/buildkit/buildkitd.toml — GC policies, worker backends, registry mirrors.
• Use TLS with client certs for multi-tenant daemons shared across a build cluster.
• Enable rootless mode with user namespaces and fuse-overlayfs for untrusted CI environments.
• Integrate with containerd by pointing --oci-worker-snapshotter=stargz for lazy image pulls.

Key Features

• Automatic parallelization of Dockerfile stages and unrelated RUN steps.
• --mount=type=cache for persistent package-manager caches across builds.
• Inline and registry-based remote cache with layer-level granularity.
• Secrets and SSH forwarding that never land in the final image or cache layers.
• SBOM and provenance attestations (SLSA) exported alongside the image.

Comparison with Similar Tools

• Docker legacy builder — BuildKit is the successor; single-threaded, no multi-arch, obsolete cache.
• Kaniko — Builds inside Kubernetes without a daemon; slower and less cache-efficient than BuildKit.
• img — Rootless builder using BuildKit internally; mostly superseded by buildx.
• Buildah — Scriptable OCI builder from the Podman family; different model, no DAG scheduler.
• nerdctl build — Uses BuildKit under the hood for containerd-native CLI workflows.

Key Flags

• --export-cache type=registry,ref=myreg/cache:latest for shared team cache.
• --attest type=sbom / type=provenance to meet supply-chain policies.
• --platform linux/amd64,linux/arm64 for multi-arch from a single Dockerfile.

FAQ

Q: Do I need BuildKit to use docker build? A: Modern Docker Desktop already enables BuildKit by default; on Linux set DOCKER_BUILDKIT=1 or install buildx.

Q: Can it build without root? A: Yes — buildkitd --rootless uses user namespaces and fuse-overlayfs; ideal for untrusted CI runners.

Q: How is the cache shared across CI agents? A: Export to a registry with type=registry and import on the next run; layer digests are content-addressable.

Q: Does it support Windows containers? A: Linux-first. Windows container builds still use the classic builder or Docker Engine''s Windows backend.

Sources

• https://github.com/moby/buildkit
• https://docs.docker.com/build/buildkit/