One of the challenges with supply chain security is ensuring that all of the code you are using is trusted. Traditionally, this primarily focused on third-party code. However, in some cases, it can also be important to ensure that your own code is trusted. As developers, we may sign our code to verify authorship since Git records the email address of the author. By signing our commits, we attest that we authored the code. There are other times where we won’t want to rely on our own credentials. We need to treat the code like it is an artifact, so we need the CI/CD process to attest that the code is unmodified. Let’s understand this use case a bit better, and see how one way we can support attestation using commit signing.
Code as the artifact
One of the most common cases for needing to treat code as an artifact is GitOps. In that case, tools like ArgoCD or Flux will treat Git repositories as a source of truth. They will contain the configuration and deployment details. While the related images may be signed, the source code itself is actually part of what is deployed. Since this repository used for deployment, it is for all purposes a deployment artifact. Changes made to the Kubernetes manifests need to go through the traditional development, test, and deploy process. Only once the changes are verified should they be deployable. This often creates a process that looks like this:
In this model, the final Git repository (“Manifest Repo”) acts similar to a package management system. The final code is pushed to that repository, and that code should represent the new system state. Kubernetes will then pull that code, apply it to the cluster, and update the system state. This assumes that the code in the repository can be trusted and hasn’t received any additional, unapproved changes. To make this secure, you need a way to ensure they can be trusted. Otherwise, anything added to the repository – no matter how it is added – can be deployed. This is where Gitsign comes in.
What is Gitsign?
Gitsign is a tool that supports signing Git commits and tags using Sigstore. The signature it creates attests to the source of the code and the process that is creating the commit (its provenance). The commit is made using a token that has the authority to claim that this is truthful, accurate information. To support that, it uses an append-only log that documents the attestation occured and the related signature for verification. This prevents tampering with the commit at a later time.
Under the covers, it does a few things:
- Creates a short-lived X.509 certificate containing commit metadata
- Uses the certificate to sign the commit
- Publishes the signature to a transparency log to allow future verification
This methodology is called ‘keyless signing’ because it does not require a long-lived private key. Instead, the signing process uses a certificate generated specifically for the commit. The certificate is created using an OIDC token, and properties from that token are stored as part of the certificate. The signature on the certificate prevents those properties from being modified after the certificate is generated. It also provides an attestation that those were the properties provided at the time the commit was created. With GitHub Actions, the Actions OIDC token is used to generate the certificate. That token incorporates details about the workflow run, the commit, and the current repository. When the commit is signed, it will include this information. Because all of these details are part of the commit, they can be incorporated into policies that use those as criteria to trust the committed code.
Signing the Commit
To Use Gitsign, you first need to install it. There are three ways to do this in Actions. The first way is to download the binary from the GitHub releases page and use chmod +x
to make it executable. The second way is to download and compile the Go source code. The final way is to use curl
or wget
to download and install the .deb
file. This way has the advantage of installing the dependencies as well. To do that in bash
:
1wget -q https://github.com/sigstore/gitsign/releases/download/v0.13.0/gitsign_0.13.0_linux_amd64.deb
2sudo dpkg -i gitsign_0.13.0_linux_amd64.deb
Once Gitsign is installed, you you need to configure Git to use it. This means configuring it as the program that is used for X.509 signing. This requires making a few changes to the local Git config:
1# Enable commit signing
2git config --local commit.gpgsign true
3
4# Enable tag signing
5git config --local tag.gpgsign true
6
7# Use the X.509 format for signing Git commits and tags
8git config --local gpg.format x509
9
10# Configure gitsign as the signing program that will be used for
11# signing commits and tags. When a commit is signed or verified,
12# gitsign will be called with a specific set of arguments.
13git config --local gpg.x509.program gitsign
Of course, if you’re going push the commit, you also need to specify the user identity that will be used. This is done using the user.name
and user.email
properties.
1git config --local user.name "Automated Commits"
2git config --local user.email "bot@users.noreply.github.com"
These values can normally be anything, but you will want to use them consistently since they represent the identity that you’re claiming created the commit. In many cases, you may prefer to push it as if it was part of the Actions process itself using the built-in token. This makes it look like the Actions bot created the commit. This makes it clear it was an automated part of an Actions process:
1git config --local user.name "https://github.com/${{github.workflow_ref}}"
2git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
With Gitsign, the it is preferable to specify a user.name
value that matches the certificates Subject Alternative Name (SAN). That will be a URI in the format https://github.com/${{github.workflow_ref}}
. That enables you to configure git config gitsign.matchCommitter true
. This enables
committer verification to ensure that the commit is signed by the same identity as the certificate. This adds an additional layer of security to the commit.
If you want even more protection, you can use a time-stamp authority to attest to the time when the signature was created. Enabling
timestamp support is done with an additional git
config setting that specifies the time server to use:
1git config --local gitsign.timestampServerURL "http://timestamp.digicert.com"
With all of those steps completed, you can create a commit and push it to the repository. Creating the commit will automatically trigger the signing. I’ll demonstrate with an empty commit, but you can use git add
to create a traditional commit as well.
1git commit -v --allow-empty --message="Signing this commit"
2git push
The generated commit will now have additional content that provides the details from the OIDC token, the certificate used for signing the commit, and a signature that can be verified.
To do this in Actions, your workflow will need two permissions: contents: write
(to read and modify the repository) and id-token: write
(to create the token used in generating the certificate).
Verifying the commit
After the commit is made, you can see the results by using git log --show-signature
. This will show the commit details, including the signature. The output will look something like this:
1git log --show-signature
2commit 0b55e229458b4ba7a504d41b305c540bf1 (HEAD -> main, origin/main, origin/HEAD)
3tlog index: 156816657
4gitsign: Signature made using certificate ID 0xede0a8bdf0c1abe49f09b755cd480adde02888b3 | CN=sigstore-intermediate,O=sigstore.dev
5gitsign: Good signature from [https://github.com/kenmuse/myrepo/.github/workflows/publish.yml@refs/heads/main](https://token.actions.githubusercontent.com)
6Validated Git signature: true
7Validated Rekor entry: true
8Validated Certificate claims: false
9WARNING: git verify-commit does not verify cert claims. Prefer using `gitsign verify` instead.
Normally, we would use git verify-commit
to verify the signature. As you can see in the logs, this provides some validation, but it does not validate the certificate claims. I’ll explain those more in a future post.
In this example, there are a few important details from the claim that are shown in the logs:
- Repository:
https://github.com/kenmuse/myrepo
- Workflow:
.github/workflows/publish.yml
- Ref:
refs/heads/main
- Branch:
main
- OIDC token issuer:
https://token.actions.githubusercontent.com
These details were claims were part of the token provided by GitHub, and they were incorporated into the certificate. Because Git only verifies the signature itself, it is unable to validate that these properties or that they are the expected values. This leads to the warning message.
Because we know the OIDC token issuer and the expected provenance for the commit, we can provide these values to Gitsign and have it verify the commit. This ensures the commit originated from the expected source. I just need to provide the commit ID (or HEAD
) and the expected values:
1gitsign verify --certificate-identity="https://github.com/kenmuse/myrepo/.github/workflows/publish.yml@refs/heads/main" --certificate-oidc-issuer=https://token.actions.githubusercontent.com 0b55e229458b4ba7a504d41b305c540bf1
Now Gitsign can can ensure that the commit was generated from the expected source. If it matches, the output will look like this:
1tlog index: 156816657
2gitsign: Signature made using certificate ID 0xede0a8bdf0c1abe49f09b755cd480adde02888b3 | CN=sigstore-intermediate,O=sigstore.dev
3gitsign: Good signature from [https://github.com/kenmuse/myrepo/.github/workflows/publish.yml@refs/heads/main](https://token.actions.githubusercontent.com)
4Validated Git signature: true
5Validated Rekor entry: true
6Validated Certificate claims: true
Notice that it has not validated the certificate claims. What happens if the commit’s provenance doesn’t match what we expect? In that case, the verification will fail:
1gitsign verify --certificate-identity="https://github.com/kenmuse/wrong-repo/.github/workflows/different.yml@refs/heads/main" --certificate-oidc-issuer=https://token.actions.githubusercontent.com 0b55e229458b4ba7a504d41b305c540bf1
2
3Error: none of the expected identities matched what was in the certificate, got subjects [https://github.com/kenmuse/myrepo/.github/workflows/publish.yml@refs/heads/main] with issuer https://token.actions.githubusercontent.com
Failing if the values don’t match is a good thing. It means that the commit was not generated from the expected source. The repo now contains code that is from an unexpected source. You no longer can trust it – the contents may have been compromised. This is the basis of supply chain security. We want to prove that the commits are trusted and originating from the expected source. Trust should be earned, not immediately provided!
The current limitations
This is an area where the industry is slowly catching up to the realities of modern software security. As a result, everything is still evolving. At the time of this writing, Gitsign is still in its early stages, so there are a few limitations:
- Transparency Logging
- Gitsign uses Rekor for transparency logging. With transparency logging, commit signatures are published to an update-only public log. While you can use private Rekor servers for private and internal repositories, there’s currently no way to disable this logging (there’s an open GitHub issue).
- GitOps Integration
- Some of the popular GitOps tools are limited to only PGP commit signatures today. ArgoCD currently has an open issue to add X.509 support. Flux relies on a third-party package, and they are waiting for SSH and X.509 support in go-git to add support.
- GitHub UI Integration
- The GitHub web interface verification status doesn’t work they way you might expect. It shows the commit has “Unverified”. Clicking on that will have one of two behaviors.
For email addresses not associated with a GitHub App or user, you’ll be warned about the missing user details:
If the email address for the commit can be associated, it will warn you that the certificate could not ve verified and display the common name for the certificate.
Security is an evolving space, so I expect the options will continue to change. Lots of tools are being updated to provide better support for attestation and supply chain security. In addition, because developers are becoming aware of these security needs, I expect the options will continue to grow and mature.