At OmbuLabs, we have some projects where multiple teams work at the same time on different features or fixes. We started using Heroku’s Review Apps opens a new window because we kept running into blockers when a team needed to deploy a branch to our staging server but another team was using it.

There are two configurations in Heroku to create Review Apps: manual and automatic. A manual creation gives us more control, but not every person involved in the QA process has access to the Heroku pipeline. So, for many months, we used the automatic Review App creation every time a PR was created/updated. This was an easy workaround, but there’s one problem, the Review Apps for Heroku Teams can’t use free dynos, so we were being charged for Review Apps that were created before they were actually needed or even for PRs that didn’t really need a Review App at all.

We started looking for an easy way to control the creation and deletion of Review Apps that can be triggered by anyone directly from the GitHub PR and here are the details and how we do this now.

Heroku’s Review Apps

For those not familiar, Heroku has a feature called Review Apps: short-lived applications that are deployed, independently, based on the active pull requests. The purpose is to help multiple teams review different pull requests in live urls without sharing a single staging or test environment. There are 3 main methods to create Review Apps:

• Automatic creation: every time a PR is created, Heroku creates a new app automatically
• Manual creation using the Dashboard: Heroku will list all the open PRs in the pipeline view, showing a Create review app button
• Using the Heroku API: Heroku provides an API opens a new window to manage the Review Apps

There are many methods to destroy a Review App:

• Stale PRs: in the Review Apps Dashboard, there’s a configuration to automatically destroy Review Apps that are stale for a set number of days
• Manual destroy using the Dashboard: every running Review App shows a Delete review app button in the pipeline screen in the Dashboard
• Using the Heroku API: Heroku provides an API to manage the Review Apps
• Merged PR: when a PR is merged, the Review App is automatically destroyed

Note that, to use the workflow we’ll describe here, you need to be sure you are using the new version of the Review Apps (introduced on November 7th 2019). If you are using the old version, Heroku will show a message prompting you to upgrade.

The QA Workflow

Before explaining the different parts, let’s see the current QA workflow we have in place, so the next sections are more clear with this context.

The workflow goes like this:

1. When a PR is ready for QA, one person is assigned to the PR on GitHub.
2. When that person is ready to QA the changes, they add the create-review-app label.
3. After a moment, a GitHub Action is executed: it calls the Heroku API to create a Review App for the current branch and adds some information to the activity log showing that the label was automatically removed.
4. After a moment, Heroku will update the activity showing that a deploy is triggered and it’s currently Pending.
5. When the deploy is done, Heroku updates the Deploys section of the PR with the link to the Review App.
6. When the QA person finishes doing the QA, they add the destroy-review-app label.
7. After a moment, a GitHub Action is executed: it calls the Heroku API to delete the Review App for the current branch and adds some information to the activity log showing that the label was automatically removed.

Note that, if the PR is merged, the Review App is destroyed automatically by Heroku

Keep in mind that, if the PR has merge conflicts, the GitHub Actions are not executed

GitHub Actions

GitHub Actions opens a new window allows adding automations to a GitHub repository. Actions can be created as GitHub repositories with a specific action.yml file that tells GitHub how to run it, more details here opens a new window . There are many events that can trigger the execution of an Action, we only use the labeled and the closed events for the workflow described before, but here’s a list opens a new window of all available events.

Manage Heroku Review Apps (GitHub Action)

We created this GitHub Action opens a new window that can be used in any project to either create or destroy Review Apps based on the current Pull Request that triggers it.

Create Review App

To create a Review App, a step can be added with this configuration:

- uses: fastruby/manage-heroku-review-app@v1.2
  with:
    action: create
  env:
    HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
    HEROKU_PIPELINE_ID: ${{ secrets.HEROKU_PIPELINE_ID }}
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

To trigger the creation of the Review App when a label is added, we use this configuration:

name: Create Review App
on:
  pull_request:
    types: [labeled]

jobs:
  create-review-app:
    if: ${{ github.event.label.name == 'create-review-app' }}
    runs-on: ubuntu-latest

    steps:
      - uses: fastruby/manage-heroku-review-app@v1.2
        with:
          action: create
        env:
          HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
          HEROKU_PIPELINE_ID: ${{ secrets.HEROKU_PIPELINE_ID }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Check the Requirements section opens a new window of the GitHub Action’s README to generate and configure the needed secrets.

Note that the action doesn’t check the user’s privileges on the repository, we don’t recommend using this in a public repository if you are worried that unwanted users will trigger the Review App creation/deletion.

Destroy Review App

Similar to the creation, to destroy a Review App we use the same GitHub Action with the action parameter set as destroy

name: Destroy Review App
on:
  pull_request:
    types: [labeled]

jobs:
  destroy-review-app:
    if: ${{ github.event.label.name == 'destroy-review-app' }}
    runs-on: ubuntu-latest

    steps:
      - uses: fastruby/manage-heroku-review-app@v1.2
        with:
          action: destroy
        env:
          HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
          HEROKU_PIPELINE_ID: ${{ secrets.HEROKU_PIPELINE_ID }}

Note that destroying a Review App doesn’t need the GITHUB_TOKEN secret.

Pull Request Un-labeler (GitHub Action)

As described in the workflow before, after adding the label, we automatically remove it once the actions are executed. To do so, we also created this GitHub Action opens a new window that can be used in a similar way indicating the label that we want to remove from a PR.

For example, if we want to remove a ready-to-qa label when a ready-to-merge label is added, we can use this configuration:

name: Unlabel
on:
  pull_request:
    types: [labeled]

jobs:
  ready-to-merge:
    if: ${{ github.event.label.name == 'ready-to-merge' }}
    runs-on: ubuntu-latest

    steps:
      - uses: fastruby/pr-unlabeler@v1
        with:
          label-to-remove: "ready-to-qa"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

In our project, to remove the label that was just added (as described in the workflow before), we use the same label in the if and for the label-to-remove parameter.

Final Config File

Putting it all together, we use this config file:

# .github/workflows/review-app-on-label.yml

name: Heroku Review App on label
on:
  pull_request:
    types: [labeled]

jobs:
  create-review-app:
    if: ${{ github.event.label.name == 'create-review-app' }}
    runs-on: ubuntu-latest

    steps:
      - uses: fastruby/manage-heroku-review-app@v1.2
        with:
          action: create
        env:
          HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
          HEROKU_PIPELINE_ID: ${{ secrets.HEROKU_PIPELINE_ID }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - uses: fastruby/pr-unlabeler@v1
        with:
          label-to-remove: "create-review-app"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  destroy-review-app:
    if: ${{ github.event.label.name == 'destroy-review-app' }}
    runs-on: ubuntu-latest

    steps:
      - uses: fastruby/manage-heroku-review-app@v1.2
        with:
          action: destroy
        env:
          HEROKU_API_TOKEN: ${{ secrets.HEROKU_API_TOKEN }}
          HEROKU_PIPELINE_ID: ${{ secrets.HEROKU_PIPELINE_ID }}

      - uses: fastruby/pr-unlabeler@v1
        with:
          label-to-remove: "destroy-review-app"
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This configuration takes care of the creation/deletion of Review Apps when the labels are added.

We also configured Heroku to destroy stale Review Apps after 1 day, in case somebody forgets to destroy it manually after QA.

In Conclusion

With this configuration we have many benefits:

• We can save money by not creating a Review App for a PR until it’s actually needed.
• We can save money by not creating a Review App for a PR that doesn’t really need one.
• We can save money by destroying the Review App as soon as the QA is done.
• There’s less context switch: the QA person doesn’t have to move between GitHub and Heroku during the QA process (we avoid having to re-login on Heroku every day!).
• We have a log of who triggered these actions in the PR’s activity log.
• And the most important for us: We enable any user with access to the repo to generate a Review App when needed, they don’t need to have access to Heroku’s Dashboard nor permissions to manage the Review Apps nor remember the process to do it manually.

The GitHub Actions were created as open source, if you like this idea and want to use it for your projects please share your comments with us!