Creating a Custom Buildroot Package For Binaries

Being able to create images and file systems with Buildroot is great, but the out-of-the-box features won’t handle situations where you need a custom set of binaries or code to be deployed as part of the image. For that, you have two options:

• Add the files to an overlay
• Create a custom package

We briefly touched on the first topic in an earlier post. In this post, I’ll dive into creating a custom package. The first thing to know is that Buildroot is really designed for creating a complete system from code, compiling each component when it is needed so that you control the configuration and dependencies for your distribution. In this case, I want to explore handling a set of binaries that were already generated as part of a build process. For this example, you’ll package and distribute the GitHub CLI as part of your custom image.

I’ll continue with the steps from the earlier posts so you can focus on the details for this package. I’ll also assume that the directory source contains the external tree with your custom configurations and output contains the generated working code. The main config uses the name MYPROJECT.

Getting started

The majority of this process has very specific conventions, so stick to those to ensure the right behaviors. Inside of source, you’ll create a folder called package. Inside of that, you’ll create a folder for each custom package you create. In this case, ghcli.

1mkdir -p source/package/ghcli
2touch source/package/ghcli/Config.in
3touch source/package/ghcli/ghcli.mk

It’s very important to know that the name of the folder dictates a number of other names:

• The package will be exposed as ghcli
• The make file for the package must be called ghcli.mk
• The package itself will be configured as BR2_PACKAGE_GHCLI (with the name in uppercase and special characters replaced with an underscore)

These conventions are strict. If they are not followed, your package may not get properly included.

To make sure it is available to the menu system, you need to add a line to the file source/Config.in that points to the new package. This configuration does not support wild cards, so you need to provide it an explicit path. Thankfully, you can at least us the special variable that Buildroot provides based on the name in source/external.desc file, MYPROJECT:

1source "$BR2_EXTERNAL_MYPROJECT_PATH/package/ghcli/Config.in"

You also need to update source/external.mk to point to your package. Thankfully, that does support wildcards. As a result, your can capture all possible packages with one line:

1include $(sort $(wildcard $(BR2_EXTERNAL_MYPROJECT_PATH)/package/*/*.mk))

With that scaffolding in place, you can now start on the package.

Defining the Kconfig

The file source/package/ghcli/Config.in is a Kconfig configuration file, part of a configuration system developed for the Linux Kernel. It starts with defining the name of the package parameter using BR2_PACKAGE_{PACKAGE_NAME} (uppercase). Since the package name is ghcli, the config name is BR2_PACKAGE_GHCLI. It will be displayed in the menu interface as “GitHub CLI”, and it will use a bool value to enable/disable the package. In a future article, I’ll discuss how you can add additional configuration options. The full configuration looks like this (with tab indentations):

1config BR2_PACKAGE_GHCLI
2    bool "GitHub CLI"
3    depends on BR2_arm || BR2_aarch64 || BR2_x86_64
4    select BR2_PACKAGE_CA_CERTIFICATES
5    help
6        Includes the GitHub CLI binary

The depends on line is used to limit your package to only systems that are configured to target ARM, aarch64, or x86/64 systems. If the Buildroot project is targeting any other platforms, depends on will cause this package to be hidden and unavailable unless you select a supported platform.

The select ensures that you have appropriate CA certificates and crypto support. The GH CLI requires that in order to securely connect to GitHub via HTTPS. Unlike depends, this actually causes a package to be automatically selected (set to y) if this package is selected. The package cannot be unselected as long as this package is enabled. Use the select option when a dependency is not obvious and you want to make sure it is automatically selected for the user.

This language has a lot of other features available. If you want to learn more, you can read about the Kconfig language on kernel.org.

Crafting the Makefile

The last step in defining a package is to populate ghcli.mk. You will need to define a few variables. The variables always start with the uppercase version of the package name:

Normally, you want to download, build, and install binaries using a source code bundle. For this example, you’ll be starting with the compiled binaries for GH CLI instead. As a result, you’ll have to explicitly detail which binaries to install and where. Buildroot will handle everything else, including downloading and unpacking the source bundle into the working directory. To do that, you need two things:

• Set GHCLI_INSTALL_TARGET = YES.
• Provide a define called GHCLI_INSTALL_TARGET_CMDS that contains the specific steps for copying the binary from the working directory (@D) to the $TARGET_DIR.

The GH CLI binaries are available from https://github.com/cli/cli/releases/download/v${GHCLI_VERSION}/gh_${GHCLI_VERSION}_linux_${GHCLI_ARCH}.tar.gz, where the GHCLI_ARCH is either amd64 or arm64. The current version is 2.75.0, so you can use that as the GHCLI_VERSION. For the architecture, you can read a special variable BR2_ARCH to understand what architecture is being built. Putting that all together:

 1##########################
 2# GitHub CLI (ghcli)
 3##########################
 4
 5# Required configuration settings
 6GHCLI_VERSION = 2.75.0
 7GHCLI_SITE = https://github.com/cli/cli/releases/download/v${GHCLI_VERSION}
 8GHCLI_SOURCE = gh_${GHCLI_VERSION}_linux_${GHCLI_ARCH}.tar.gz
 9GHCLI_INSTALL_TARGET = YES
10
11# Determine the architecture target
12ifeq ($(BR2_ARCH), "aarch64")
13    GHCLI_ARCH=arm64
14endif
15ifeq ($(BR2_ARCH), "arm")
16    GHCLI_ARCH=arm64
17endif
18ifeq ($(BR2_ARCH), "x86_64")
19    GHCLI_ARCH=amd64
20endif
21
22# Define the installation process
23define GHCLI_INSTALL_TARGET_CMDS
24    # Write a friendly message
25    $(info Installing GH CLI $(GHCLI_VERSION) for $(GHCLI_ARCH))
26
27    # Copy the pre-compiled binary from the working directory to the target
28    $(INSTALL) -D -m 0755 $(@D)/bin/gh $(TARGET_DIR)/usr/local/bin/gh
29endef
30
31$(eval $(generic-package))

The custom define echos a short message to the console, then uses the Linux install command to copy the gh binary to the target directory with the appropriate permissions. Yes, you could have used cp and chmod instead, but install does it all in one command. In addition, Buildroot has create a custom variable contains the full path to the install command, so you can use that as a helper.

At the end, you need to call the eval block to allow Buildroot to process all of the values you configured. This automates a number of commands using the handful of definitions you provided in the make file. The extra empty line after that (at the end of the file) is important, so don’t forget it!

If you're new to the world of Make, no worries! There's a nice tutorial at https://makefiletutorial.com/ that can teach you all of the key details. For Buildroot, you're generally following strict conventions, then invoking a small number of shell commands. In this case, just enough to copy the binaries.

The other methods

You aren’t limited to just downloading an HTTPS package. Buildroot is very configurable and supports a few different options natively. For example, if a git:// URL is used, the version can be a release tarball, SHA1, or tag (branches aren’t supported). You can specify an approach by adding a variable, {PACKAGE}_SITE_METHOD that indicates the approach to use:

• local for local tarball with {PACKAGE}_SITE specifying a local directory path
• file for a package tarball (specified using {PACKAGE}_SITE)
• svn, cvs, git, hg, or bzr for source control repositories
• scp and sftp for SSH based file transfers
• smb to use `curl to retrieve the file from a share

All of these use the native tool’s approach to authentication, so you can access private data stores. You can read about the options in the Buildroot manual, generic package reference.

Verifying data integrity (optional)

You can optionally create an hash file, ghcli.hash, that contains the checksum hashes for the binaries that you are downloading. When this file is present, Buildroot will check the downloaded package file to determine whether or not the download is unmodified. The file contains three space-delimited columns: the hash type, the hash value, and the downloaded file name. For GH CLI, you can save some time in calculating these. The values are provided as part of the release: https://github.com/cli/cli/releases/download/v2.75.0/gh_2.75.0_checksums.txt. You can download the file and copy out the appropriate values. The final file would look like this:

1sha256 98f910f4cde42a345e241f867f42b2229dda60267d7f87a56d283842519a4cb7 gh_2.75.0_linux_amd64.tar.gz
2sha256 353d8769b963b2929266e2d84237dc60061a6922cc901608e73b9fd776c3cade gh_2.75.0_linux_arm64.tar.gz

Now, Buildroot will automatically ensure that the package has not been modified or corrupted.

Testing the package

With everything properly configured, you have a file structure that looks like this:

  source
  ├── Config.in
  ├── configs
  │   └── base_defconfig
  ├── external.desc
  ├── external.mk
  └── package
      └── ghcli
          ├── Config.in
          ├── ghcli.hash
          └── ghcli.mk

To test the package, run make -C output nconfig. Then select External options. You should see a screen similar to this. Notice that the new package is available and ready to be selected!

 ┌─── External options ────────────────────────────────────────────────────────┐
 │                                                                             │
 │    *** My custom Buildroot project (in /workspaces/myproject/source) ***    │
 │    [ ] GitHub CLI                                                           │
 │                                                                             │

Select the package and enable it, then try running make -C output -j$(nproc). Your new image should have the GH CLI binary available and ready to use.

Your first package

As you can see, it’s easy to incorporate a custom package into a Buildroot project, and it can be incredibly simple to add a package that just downloads and installs a particular binary. Of course, you might want to make your package more configurable. For example, what if you want to allow for a custom version number? What if you want to make it a configurable option to include the man pages that are in the GH CLI release package? Feel free to experiment while I put together a future post on the subject!