Speeding Up Docker Build Times

In the rapid environment of software development, time is crucial. If you've experienced the frustration of waiting for Docker builds to complete, you're in good company.

This article introduces a range of strategies and techniques designed to accelerate Docker builds significantly and empower you to better manage your development workflow.

Strategies and Techniques covered

• Caching image layers
• Caching app dependencies using cache mount type
• Using .dockerignore
• Parallelization (using Docker and using Gradle)

Caching Image Layers

A highly effective way to speed up your Docker builds is by utilizing Docker layer caching. This feature enables the caching of intermediate build layers, so you don’t need to rebuild them from scratch each time you modify your code. The official Docker documentation offers an excellent guide on this subject. Since Docker cache management is the basis for the upcoming techniques, I suggest reading it thoroughly and even trying out the concepts in a simple exercise.

Caching App Dependencies Using Cache Mount Type

Most build frameworks offer a caching mechanism for application dependencies, which helps optimize subsequent builds. However, when used within a Docker build, the build framework cache is always recreated from scratch, as the build starts with a fresh base image.

In the example below, we’ll use Gradle as the primary build framework to compile the necessary binaries for the application, which will then be packaged into a Docker image. That said, the 'cache mount type' approach is applicable to any build framework that provides a dependency caching mechanism.

This approach requires using BuildKit as the Docker backend, an improved system that replaces the legacy builder. If you’re not already using BuildKit, refer to the provided link to enable it.

The key feature we’ll take advantage of from the BuildKit backend is the --mount=type=cache option.

For this example, we’ll use the Red Hat Universal Base Image as the base image in our Dockerfile. Additionally, we’ll assume that the application's source code is located in the same directory as its Dockerfile.

# syntax=docker/dockerfile:1
FROM redhat/ubi9:9.2

# Prerequisites setup (e.g. java)
# ...

# Copy the application source code
COPY . /source

# Build application and mount the Gradle cache located under /root/.gradle ($HOME/.gradle)
RUN --mount=type=cache,target=/root/.gradle cd /source/ && ./gradlew build

# Rest of Dockerfile content (e.g. a new runtime stage)
# ...

In the snippet above, we run the required build task for the application (gradlew build) while also directing Docker to cache the Gradle user home directory, which contains the dependencies cache (/root/.gradle). Keep in mind that the location of the Gradle user home directory can vary depending on the user.

The first Docker build will cache the Gradle dependencies, leading to much faster subsequent builds, especially for applications with many dependencies. This is achieved by adding --mount=type=cache,target=/root/.gradle to an existing RUN instruction in your Dockerfile.

Using .dockerignore

The .dockerignore file functions similarly to the .gitignore file, allowing you to specify which files or directories Docker should exclude during the build process. It is vital for reducing image size and considerably speeding up the build process, particularly during the loading of the build context. It’s important to exclude the following items during Docker build execution:

• All build/staging directories — locally built artifacts will be recreated within the Docker build process
• The .git directory — potentially very large and unnecessary (for the build process) directory that can bloat the Docker build context
• Any other large files or directories that are not required for the build but are part of your repository

Here’s an example of a .dockerignore file that illustrates the three points mentioned above:

**/build
build
.git
other/not/required/things/mordor

Parallelization

In this section, we’ll explore two methods for parallelizing builds:

• Using Docker itself
• Leveraging the build framework, in this case Gradle

In the 'Caching App Dependencies Using Cache Mount Type' section above, we utilized a powerful feature of the BuildKit Docker backend. By switching to BuildKit, you automatically gain the ability to parallelize subsequent Docker builds. BuildKit can execute independent build steps in parallel when possible and optimize away commands that don’t impact the final result.

The other approach makes use of the Gradle build framework’s parallel mode. By using the --parallel flag, Gradle can run tasks in parallel, as long as they belong to separate projects. Enabling parallel builds can significantly reduce build times, with the extent of improvement depending on your project's structure and the number of dependencies between tasks.

Now, let's revisit our previous Dockerfile example and incorporate the --parallel flag:

# syntax=docker/dockerfile:1
FROM redhat/ubi9:9.2

# Prerequisites setup (e.g. java)
# ...

# Copy the application source code
COPY . /source

# Build application and mount the Gradle cache located under /root/.gradle ($HOME/.gradle)
RUN --mount=type=cache,target=/root/.gradle cd /source/ && ./gradlew build --parallel

# Rest of Dockerfile content (e.g. a new runtime stage)
# ...

Note that if build errors are encountered when using the --parallel switch compared to non-parallel execution, your build scripts may need further adjustments by adding necessary task dependencies.

Wrapping Up

In this article, we've covered four simple techniques to speed up your Docker build times. In conclusion, it's evident that BuildKit is the key to unlocking the majority of these optimizations and boosting your productivity.