Ken Muse

The Two GitHub ARCs


If you’re new to GitHub’s Actions Runner Controller (ARC), you may not realize there are actually two different versions of that product. Because of a change in direction earlier this year, there’s naturally a bit of confusion in the community. I’m going to try to clear that up.

First things first, this is not the same thing as Azure Arc. That’s a very different product. It allows you to manage on-premises workloads using Azure’s management tools. It allows you to manage your cloud and datacenter from one place. It also has specific feature for managing Kubernetes clusters, including the ability to deploy select Azure services onto a self-maintained cluster. For example, you can run App Services, Logic Apps, and Azure SQL Managed Instance. This is not what I’ll be discussing.

GitHub ARC comes in two flavors: legacy and “autoscaling runner scale sets” (modern).

The Legacy edition

Community ARC (or “legacy” ARC) is the original product. It is an open-source project designed to help manage and scale self-hosted runners on Kubernetes. It eventually attracted attention from GitHub. This led to GitHub helping to sponsor the project, including providing internal resources to help with the development. Ultimately, this led to the project being moved into the Actions organization.

This version of ARC is still being managed and developed by the community. It’s an incredibly feature-rich platform, allowing you numerous configuration options. It generally encompasses every component that is part of the actions.summerwind.net namespace. If you do a search for GitHub ARC (or for Actions Runner Controller), you may find yourself in the docs folder within the ARC repository. That’s documentation for the legacy features. You’ll notice there’s a lot of it! The legacy version was powerful and complex, so it required quite a bit of documentation. Not all of the documentation was in the repo’s markdown. For example, customizing the runner meant reviewing that code. There was a lot of convention involved, so creating a working runner from scratch meant understanding the implementation and assumptions.

Issues with this version of ARC are no longer handled by GitHub Support. You can raise issues in the repository and get support from the user community. Longer term,

The Modern edition

The modern version of ARC (Autoscaling Runner Scale Sets mode) was released in July 2023 and is the version supported by GitHub Support. For example, it removes the dependency on cert-manager. ARC also has a simpler setup process, and its new Helm charts are published as OCI packages rather than tarballs. It also takes advantage of a newer API designed to minimize issues with rate limiting (a common issue for users of the legacy ARC). It exists in a new namespace (actions.github.com), and the documentation is managed on docs.github.com.

One thing that makes this a bit confusing is that both projects are open source and share a common code base in the same repository. This often results in teams inadvertently finding themselves reading documentation, issues, or discussions from the legacy version. As long as you’re aware of the issue (and the new documentation location), you can often avoid it.

Modern ARC has the same features as the legacy platform (and more), but includes some important design changes intended to make it easier for teams to adopt. In fact, a basic deployment involves running a Helm chart for the controller and then configuring and deploying a Helm chart for each scale set. Most users can set up a basic environment with minimal effort in a matter of minutes.

The runner image is also simpler. The default configuration expects a specific folder on the runner (and some supporting scripts). It has a minimal setup, making it ideal as a base image. This also makes it easy to create your own image from scratch. The base image is now managed in the Actions Runner repository. That means it is also automatically updated when the runner codebase changes. The latest image always has the most up-to-date runner.

There is a bit more to understand if you create a custom image and need to support job containers, containerized Action, or container services. Those features require a set of scripts —- container hooks —- that are responsible for processing those requests. The default image includes the necessary scripts, providing support for both Docker-in-Docker and Kubernetes mode. The two modes are covered in the ARC documentation. If you’re building an image from scratch, you’ll want to include (and possibly customize) those scripts.

This version of ARC is designed and optimized for Linux runners on a vanilla Kubernetes distribution. At the moment, GitHub does not officially support Windows runners. While there are ways to use Windows runners on ARC, GitHub Support won’t be able to help you with issues. If you need Windows support, I strongly recommend considering the ephemeral GitHub-hosted runners. They will almost always outperform containers (and require far less maintenance and setup).

It’s also worth mentioning that OpenShift is not a vanilla Kubernetes distribution. As a result, it’s not currently officially supported by GitHub. That said, it doesn’t mean it won’t run in that environment. Since ARC is open-source (as are the runners), it’s possible to make it work in quite a few unsupported scenarios!

There is another important change in modern ARC that catches teams off guard: it doesn’t support labels. GitHub generally recommends using runner groups (or the scale set name) to select the runners being targeted by a workflow. The new larger hosted runners were the first to adopt this approach, and ARC followed that direction. In my experience, too many teams misunderstood how to use labels. This change makes it more predictable for administrators to understand which runners will be used by a given workflow.

A word of warning

Just because GitHub makes it easy to deploy and manage runners on Kubernetes does not mean that everyone should rush to move to self-hosted runners on Kubernetes. For most teams, GitHub-hosted runners are faster, more secure, and have a better SLA. Before moving to self-hosted runners, make sure you understand the tradeoffs.

ARC is GitHub’s recommended approach for ephemeral self-hosted runners. That said, ARC truly requires someone that has multiple years of experience managing, maintaining, and troubleshooting Kubernetes. It can be challenging to setup Kubernetes correctly (which includes also understanding how to manage storage, low-level permissions, and log/metrics handling). Without someone with these skills, teams will often find themselves struggling with performance and reliability issues. If you have strong experience with Kubernetes, then ARC can become a powerful tool.