Ken Muse

Creating GitHub Checks (and Understanding the Checks API)


Sometimes, a GitHub workflow needs outside help during a pull request. For example, it may rely on an external process which happens asynchronously. Perhaps it needs to invoke an external workflow to determine whether a merge should be allowed. The most common approach to solving that problem is to invoke a process within the workflow, then wait for it to complete. This approach forces a runner to sit and wait, costing time (and money).

The better solution to this problem is the Checks API.

Is it Check or Status?

Generally speaking, a Status just provides a name (called the context) and state (error, failure, pending, or success). It can optionally include a brief description and a URL. A Status appears on the Conversation tab in a pull request, and it can be configured to block a pull request. The Check is associated with a single commit. The Status API can be used to create a Status, and a Status is automatically created for each job that is run in a workflow. This is part of how a job can be used to block a pull request.

The Checks API can be thought of as an improved version of the Status API. Checks (or technically, a Check Run) can provide additional details, including line annotations, summary and detail messages (in Markdown format), a reference to an external system identifier, and images. In addition, it separates a general status (queued, in_progress, completed) from a granular state (action_required, cancelled, failure, neutral, success, skipped, or timed_out). To support these extra details, pull requests provide a dedicated tab for Checks. Similar to a Status, the overall state is displayed on the Conversation tab, and it can be used to block a pull request. Unlike a Status, it is not created as part of a job and the APIs are restricted to being called from GitHub Apps.

Checks and Workflows

Under the covers, Actions are a special type of GitHub App. Consequently, the token provided by the workflow is also an installation access token, capable of invoking the API. That means that it is possible to implement a check as part of a workflow.

There are two easy ways to create a Check from within a workflow. The first approach is to use curl to POST a JSON message to the API endpoint. To make this work, the workflow will need to use the commit SHA, github.event.pull_request.head.sha (for more details about why this is the right value, see The Many SHAs of a GitHub Pull Request).

For example:

 1- name: Create Check Run
 2  env:
 3    GH_TOKEN: ${{ github.token }}
 4  run: |
 5    curl -L -X POST \
 6     -H "Accept: application/vnd.github+json" \
 7     -H "Authorization: Bearer $GH_TOKEN"\
 8     -H "X-GitHub-Api-Version: 2022-11-28" \
 9     https://api.github.com/repos/${{ github.repository }}/check-runs \
10     -d '{"name":"A Mighty Test", "head_sha":"${{ github.event.pull_request.head.sha }}", "status":"in_progress","output":{"title":"A Mighty Summary"}}'    

With this approach, we pass the token as an environment variable so that it isn’t presented as part of the command line, further protecting the secret from accidental exposure. The context variable github.repository is used to provide the organization/owner and repository name.

The second approach is similar, but it uses the GitHub CLI:

 1- name: Create Check Run
 2  env:
 3    GH_TOKEN: ${{ github.token }}
 4  run: |
 5     gh api -X POST -H "Accept: application/vnd.github+json" \
 6      -H "X-GitHub-Api-Version: 2022-11-28" \
 7      -f 'name=A Mighty Test' \
 8      -f 'head_sha=${{ github.event.pull_request.head.sha }}' \
 9      -f 'status=in_progress' \
10      -f 'output[title]=A Mighty Summary' \
11      /repos/${{ github.repository }}/check-runs     

or if you prefer to write raw JSON instead:

 1- name: Create Check Run
 2  env:
 3    GH_TOKEN: ${{ github.token }}
 4  run: |
 5     gh api -X POST -H "Accept: application/vnd.github+json" \
 6      -H "X-GitHub-Api-Version: 2022-11-28" \
 7      /repos/${{ github.repository }}/check-runs
 8      --input - <<- EOF
 9        {
10            "name: "A Mighty Test",
11            "head_sha": "${{ github.event.pull_request.head.sha }}",
12            "status": "in_progress",
13            "output": {
14                "title": "A Mighty Summary"
15            }
16        }
17     EOF     

The status, in_progress indicates that the task has started. To indicate that the task is complete, a conclusion should be sent, typically with a value such as success or failure (which automatically sets status to completed). For tasks that complete immediately, the initial call can set the conclusion.

A Check’s name is similar to a Status context, displayed in the Status and in the Check’s tab. If multiple Checks share the same name, only the most recent one and its status will be shown in the UI. The older versions – up to 1,000 – are still available via the API. Because the name also acts as a Status context, it can be configured as required status check in a branch protection rule. This allows it to block a pull request merge. If multiple Checks exist with the same name, only the most recently updated one will be used for the Status. To ensure the Status operates as expected (and appears correctly in the UI), it’s generally a best practice to give each Check a unique name.

After calling the API, the Check object will be returned as JSON. This object will include an id field. This value can be used to update the Check by sending a PATCH message with the updated values to the /repos/{owner}/{repo}/check-runs/{id} endpoint. The easiest way to retrieve the id is using jq. Either add --jq '.id' to the GitHub CLI command or pipe the curl request output(curl {parameters}) | jq '.id').

The Check Run object

The check run object can be quite complex, and its not always intuitive where each part will be displayed. To make it easier to understand how the parts relate, the components are marked in the diagram below.

Elements of a check run
Figure 1. The elements of a check run

Building a process

While it is possible to use the Check within an Action, it’s common to use a GitHub App. This leads to one of three general approaches:

  • Create a Check inside of a workflow, then pass the identifier for the Check Run to an external system. The external system then uses the GitHub App to update the Check and indicate when it is completed.
  • Use a web hook to be notified of pull requests. Create the Check Run and update it as required.

The general workflow process:

  • If the processing is being queued or send through a web hook, the initial status is typically set to queued.
  • Once the processing request is received by the queue, send a PATCH request to update the status in_process. If the process is directly executed, consider starting with the in_process status.
  • When the process is completed, send a PATCH request with the appropriate conclusion. The status will be automatically set to completed.

During all of these phases, you can provide additional details:

  • An output object which can update throughout the process. This includes a friendly title for the check run. It can also include content for a summary or details (text). Both of those fields support 64K characters and Markdown content. The object can also include links to images and their associated and captions in images. These are appended between the summary and details. They are always scaled to 100% width. Like annotations, each request with images results in those additional images being appended to the Check Run. When updating output, the title and summary are generally required.
  • The output can also contain up to 50 annotations per update, with the annotations being appended (not replaced/updated) in each new request. Annotations can add comments to specific lines in a file and mark those as notice, warning, or failure. An appropriate icon will be associated with the annotation.
  • A collection of actions to enable automatic interactions with a GitHub App. When this is used, the conclusion is often set to action_required to add a Resolve link to the Check Run. An action object includes a label for a button (up to 20 characters), a hover description for the button (up to 40 characters), and an identifier (20 characters). When the user clicks the button, the check_run.requested_action event is raised and sent to the associated GitHub App. This event includes the repository, the Check Run object, and the identifier.

During the process, I don’t recommend changing the name of the check run; you might end up with two entries in the Checks. Instead, ignore that field for all updates.

Crossing the workflows

Normally, it’s impossible to use the GITHUB_TOKEN to invoke a GitHub API and have that trigger a workflow. GitHub prevents workflows from doing this to avoid potential infinite loops. Like all rules, this one has exceptions. A workflow can create a repository_dispatch event from within the same repository using the provided token.

I will take advantage of this to create a working Check Run process.

Putting it all together

To demonstrate the process, we’ll create two workflows in a repository. The first workflow will trigger on a PR to create a new Check Run and then invoke repository_dispatch. The second workflow will respond to the repository_dispatch event to complete the Check Run. In a typical process this would most likely all be handled using a GitHub App and web hooks; this will provide a simple example to illustrate the process. Both workflows should be committed to the default branch (main).

The pull request workflow (.github/workflows/pull_request.yml):

 1name: Create Check
 2
 3# To trigger the check
 4on:  
 5  pull_request:
 6    branches: [ "main" ]
 7
 8jobs:
 9  start-check:
10    runs-on: ubuntu-latest
11    permissions:
12      checks: write   # Permission to create a Check Run
13      contents: write # Permission to write a repository_dispatch requests
14    steps:
15      - name: Create Check
16        id: checkrun                    # An ID to allow the step to be referenced
17        env:
18          GH_TOKEN: ${{ github.token }} # Expose the token for GH CLI
19        run: |
20          ##########################################################
21          # Create a Check Run and indicate that it is being queued
22          # Use --jq to return the ID
23          ##########################################################
24
25          CHECKID=$(gh api -X POST -H "Accept: application/vnd.github+json" \
26            -H "X-GitHub-Api-Version: 2022-11-28" \
27            -f name='Super Check' \
28            -f head_sha='${{ github.event.pull_request.head.sha }}' \
29            -f status='queued' \
30            -f 'output[title]=My Check Run Title' \
31            -f 'output[summary]=A *fancy* summary' \
32            -f 'output[text]=More detailed Markdown **text**' \
33            --jq '.id' \
34            /repos/${{ github.repository }}/check-runs)
35          
36          ####################################################
37          # Put the ID into a step variable
38          ####################################################
39
40          echo "checkId=$CHECKID" >> $GITHUB_OUTPUT          
41
42      - name: Repository Dispatch
43        env:
44          GH_TOKEN: ${{ github.token }}
45        run: |
46          ##########################################################
47          # Create a repository_dispatch event of type my-check
48          # Send the SHA and the Check Run ID in the client_payload
49          ##########################################################
50
51          gh api -X POST -H "Accept: application/vnd.github+json" \
52            -H "X-GitHub-Api-Version: 2022-11-28" \
53            -f 'event_type=my-check' \
54            -f 'client_payload[checkRunId]=${{ steps.checkrun.outputs.checkId }}' \
55            -f 'client_payload[sha]=${{ github.sha }}' \
56            /repos/${{ github.repository }}/dispatches          

And for the repository dispatch (.github/workflows/dispatch.yml):

 1name: Repository Dispatch
 2on:
 3  repository_dispatch:
 4    types: [my-check]
 5jobs:
 6  myEvent:
 7    runs-on: ubuntu-latest
 8    steps:
 9      ####################################################
10      # Update the status to show that the queued message
11      # was received and is being processed
12      ####################################################
13      - name: Acknowledge Request
14        env:
15          GH_TOKEN: ${{ github.token }}
16        run: |
17          gh api -X PATCH -H "Accept: application/vnd.github+json" \
18            -H "X-GitHub-Api-Version: 2022-11-28" \
19            -f 'status=in_progress' \
20            -f 'output[title]=My Check Run Title 🤔' \
21            -f 'output[summary]=A *fancy* summary' \
22            -f 'output[images][][image_url]=https://www.kenmuse.com/favicon/apple-touch-icon.png' \
23            -f 'output[images][][caption]=Sample from kenmuse.com' \
24            -f 'output[images][][alt]=Logo for kenmuse.com' \
25            /repos/${{ github.repository }}/check-runs/${{ github.event.client_payload.checkRunId }}          
26      
27      ####################################################
28      # Actually, we'll just sleep to simulate some work
29      ####################################################
30      - name: Processing        
31        run: sleep 10
32
33      #####################################################
34      # Send a final message to complete the run and
35      # provide any final updates. Doing this one in JSON
36      # to make it more readable. This approach can also
37      # be used to get total control over the serialized
38      # data types (for example, integers).
39      #####################################################
40      - name: Complete Check
41        env:
42          GH_TOKEN: ${{ github.token }}
43        run: |
44          gh api -X PATCH -H "Accept: application/vnd.github+json" \
45            -H "X-GitHub-Api-Version: 2022-11-28" \
46            /repos/${{ github.repository }}/check-runs/${{ github.event.client_payload.checkRunId }} \
47            --input - <<- EOF
48            {
49              "conclusion": "success",
50              "details_url": "https://details.kenmuse.com",
51              "output": {
52                "title": "My Check Run Title 🚀",
53                "summary": "**Summary**: The run completed.",
54                "text": "Everything worked as expected. You should see a logo above."
55               }
56            }
57          EOF          

And there you have it – a complete, asynchronous Check implemented entirely using GitHub Actions workflows. To test it out, just create a pull request. This will trigger the two workflows and execute the Check Run. You can view the results in the Pull Request’s Checks tab. If you decide to implement something similar in a GitHub App, it’s the same process and approach. It’s simply a series of REST calls to provided the results.

Have fun implementing your own Checks!