Ken Muse

Building Base Images for ARC

If you’re using Actions Runner Controller (ARC), you may be relying on the out-of-the-box runner image, Unfortunately, that’s not likely to be enough to be successful with building on ARC. In fact, you’ll almost certainly need to build your own. Why? Because the base image is missing some components that you likely need for your processes.

There are multiple hidden dependencies in the runner that are not immediately apparent. The runner expects certain applications to be available locally. Without those apps, some parts of the runner may not operate properly. This can lead to performance issues or unexpected failures.

Let’s start with what’s provided on the runner. The runner has to be able to run scripts and commands using a shell. For Windows, that means the native cmd.exe and for Linux and macOS, bash. To run JavaScript tasks and scripts, the runner needs the node runtime. The runner can also use Dockerfiles to build images (or use OCI images directly), so it needs the docker CLI. All of these components are included in the base image (with versions of node being packaged with the runner itself). That said, there is no bundled Docker daemon, so the base image relies on a socket or mount from outside the container. The runner codebase also relies on tar and awk for some operations. Despite the .NET runtime containing support for TAR and GZIP files, the runner relies on the OS-provided utilities.

That brings us to the hidden dependencies that you need to include. It may surprise you to know that git is not included in the base image. The runner expects a minimum version of 2.0, but you will get warnings if you are using a version earlier than 2.9 on Linux or 2.14.2 on Windows. At the time of this post, the current version is 2.45.1. That’s more than 31 releases to improve the performance and fix security issues. In short, it’s a good idea to keep git up-to-date. If you are using large file storage (LFS), you will also need git-lfs version 2.9 or better. Like Git, I recommend keeping this up-to-date as well. That version was from 2019, and there have been many improvements on the path to version 3.5.1.

The runner image sets up an additional registry, git-core/ppa. This makes it easy to setup the latest versions of git and git-lfs. Simply run sudo apt update and sudo apt-get install -y git git-lfs

Beyond that, there are some binaries that support the runner’s ability to use caches, apply updates, or handle JSON responses in scripts. To support these behaviors, you’ll need to include curl, jq, and unzip. These can be included by calling sudo apt update and sudo apt-get install -y curl jq unzip in your Dockerfile. While these utilities aren’t always necessary, they ensure the runner’s scripts will all behave as expected.

There are also a few optional dependencies to also consider. If you have workflows that might need to run PowerShell scripts, then the runner will need pwsh installed. That will also provide support for .ps1 script files. It is something the runner is hardcoded to support, although PowerShell scripts are less frequently used with Linux. I also recommend including the GitHub CLI. This is the easiest way to call the GitHub APIs as part of a workflow (and often simpler and easier to use than actions/github-script).

There is an additional consideration for your base image – using Ubuntu. The base image GitHub provides uses this platform, and it is a deliberate choice. Several Actions are written to target the GitHub-hosted runners, and those use Ubuntu. For example, actions/setup-python relies on Python builds provided by The Linux versions of Python are specifically built for Ubuntu. That means that using a different operating system base can lead to unexpected issues with Actions failing.

Still thinking of using an image like Alpine because it’s smaller? It turns out that when you add the binary dependencies required for building most platforms or languages, the size difference becomes negligible. In some cases, it can make the Alpine image larger than Ubuntu. If you want to avoid issues, I recommend sticking with Ubuntu. It’s what most Actions will target.

This goes without saying, but try to keep your base image up-to-date. GitHub is constantly updating the runner and the dependencies it relies on. This ensures you have the latest security patches, performance improvements, and functionality. This also ensures that the runner version remains up-to-date, since you are required to update runners to the newest version within 30 days of a new release. If you forget to do this, the runner may be blocked from receiving jobs.

Finally, to get the best performance from the runner, you’ll want to ensure that you’re caching Actions and tools. This saves the runner from downloading the same dependencies over and over again. I also speeds up workflows and reduces the risk of hitting API limits.

What about the other components – SDKs, platforms, and other binary tools? I recommend trying to use Actions to install those components as tools. This allows you to incorporate those components as part of the Actions cache and tool cache, enables easy versioning, and ensures that the components are reusable for GitHub-hosted runners should you ever need them. It also allows developers to explicitly see the dependencies for a given workflow, rather than those being hidden in the base image. More on that in a future post.