How to deploy a Gatsby site to Netlify using Azure Pipelines

Published on

Last Updated on

Estimated Reading Time: 5 min

devops tutorial gatsby Azure pipelines

Deploying a site to Netlify is super simple and has served me well so far. But, it has a few limits, which resulted in my builds failing after my site grew to its current size.

  • a 15-minute timeout limit for builds
  • the free tier has 300 free minutes in a month. In most cases, this is fine, but gatsby takes a fair amount of time to process images, and I was about to hit the limit.

First Attempt - Cache the gatsby folders

For some time, I solved the issue by using the unofficial gatsby-plugin-netlify-cache plugin. The plugin uses an undocumented shared folder to cache the .cache and public folders across builds.

Unfortunately, this solution had 2 main issues.

  1. A clear cache and deploy site would fail.
  2. Since my site is image-heavy, I started reaching close to the limit again.

Second Attempt - Use Azure Pipelines

Azure Pipelines uses a YAML config file to build your project. We will use templates to have separate workflows for pr_and _release builds. Let's create 3 files:

  1. azure-pipelines.common.yaml: The template file with the job to build the gatsby site and deploy to Netlify.
  2. azure-pipelines.pr.yaml: The pipeline to run for PR(Pull Request) builds.
  3. azure-pipelines.ci.yaml: The pipeline to run for Live builds.

I have separate workflows for Live and PR builds because I do not want PR builds deployed to the live site.

Common Job to Build Gatsby Site

In azure-pipelines.common.yaml, we create the job to build the gatsby site.

jobs:
    - job: Build

      pool:
          vmImage: 'ubuntu-latest'

      steps:
          - checkout: self
            lfs: true

          - task: NodeTool@@0
            displayName: 'Install Node.js'
            inputs:
                versionSpec: '10.x'

          - script: yarn install
            displayName: 'Install dependencies'

          - script: yarn build
            displayName: 'Build Gatsby'

We name the job as Build and use the default ubuntu-latest image. This is followed by a series of steps.

  1. Checkout: Check out the repo and download the files from git-lfs.
  2. Install Node.js: Install Node 10.x.
  3. Install dependencies: Execute yarn install.
  4. Build Gatsby: Execute the yarn build script, which calls gatsby build.

Live Builds

Let's add the trigger for live builds in azure-pipelines.ci.yaml.

name: 'Live'

trigger:
    batch: false
    branches:
        include:
            - master

pr: none

jobs:
    - template: azure-pipelines.common.yml
  1. name: The name for this pipeline.
  2. trigger: Conditions for running the pipeline when a push is made.
    1. batch: false: trigger the pipeline for each commit, i.e. disable batching builds.
    2. branches: Include only the master branch, so the pipeline is triggered only for commits pushed to master.
  3. pr: none: Disable triggering the pipeline when a PR is opened or changes are pushed to an open PR.
  4. jobs: The jobs to run.
    1. template: Use the job template created in azure-pipelines.common.yml.

PR(Pull Requests) Builds

I also want a build to be triggered every time a PR is made against master.

name: 'PR $(System.PullRequest.PullRequestId)'

trigger:
    branches:
        exclude:
            - '*'
pr:
    autoCancel: false
    branches:
        include:
            - master

    - template: azure-pipelines.common.yml
  1. name: The name for this pipeline.
  2. trigger: Conditions for running the pipeline when a push is made.
    1. branches: Exclude all branches so that the build is not triggered for commits on any branch, including master.
  3. pr:: Conditions for running the pipeline when a PR is opened or changes pushed to an open PR.
    1. autoCancel: Additional pushes should not cancel in-progress runs.
    2. include: Include only the master branch so that the pipeline is triggered only for PR's opened against master.
  4. jobs: The jobs to run.
    1. template: Use the job template created in azure-pipelines.common.yml.

At this point, you might be wondering about having 2 separate files for pr and Live builds. This will become important in the next section when we add the steps to deploy to Netlify.

Deploying to Netlify

To deploy to Netlify, we need a few variables passed to the template as parameters and then passed as flags to the Netlify deploy command.

First, let's see what changes we need to make to the template azure-pipelines.common.yml. The updated lines are highlighted.

parameters:
    deployScriptParams: '' # defaults for any parameters that aren't specified
    netlifySiteId: 'defaultID'
    netlifyToken: 'defaultToken'
    message: ''

jobs:
    - job: Build_And_Deploy

      pool:
          vmImage: 'ubuntu-latest'

      steps:
          #unchanged from before

          - script: yarn build
            displayName: 'Build Gatsby'

          # prettier-ignore
          - script: yarn netlify deploy --site ${{ parameters.netlifySiteId }} --auth ${{ parameters.netlifyToken }} --message '${{ parameters.message }}' ${{ parameters.deployScriptParams }} --dir=public
            displayName: 'Deploy to Netlify'
  1. parameters: The environment variables to pass as flags to the Netlify deploy command.
    1. deployScriptParams: Any additional parameters that should be passed to the deploy command.
    2. netlifySiteId: The site ID for the Netlify site.
    3. netlifyToken: The access token to authorize Azure Pipeline to deploy the build to Netlify.
    4. message: The message to include in the deploy log.
  2. Deploy to Netlify: The Netlify deploy command takes the siteId, token, message and deployScriptParams as flags. We also set the dir flag to public so that the public folder is deployed.
    1. # prettier-ignore: I have added this so that prettier does not try to break up the command into multiple lines.

Now we update azure-piplelines.ci.yaml for the live build.

# unchanged from before
variables:
    - template: azure-pipelines.vars.yml

jobs:
    - template: azure-pipelines.common.yml
      parameters:
          deployScriptParams: '--prod'
          netlifySiteId: $(netlify.siteId)
          netlifyToken: $(netlify.token)
          message: 'Live $(Build.SourceVersion) $(Build.SourceVersionMessage)'
  1. variables: We set a template file to get the environment variables.
  2. parameters: The parameters to be passed to the job template
    1. deployScriptParams: I pass the --prod flag for live builds, which tells Netlify to deploy the build to production.
    2. netlifySiteId: The site ID from environment variables.
    3. netlifyToken: The token from environment variables.
    4. message: The message is created using the commit ID and the comment of the commit.

Next azure-piplelines.pr.yaml for the PR builds.

# unchanged from before
variables:
    - template: azure-pipelines.vars.yml

jobs:
    - template: azure-pipelines.common.yml
      parameters:
          netlifySiteId: $(netlify.siteId)
          netlifyToken: $(netlify.token)
          message: 'PR $(System.PullRequest.PullRequestNumber) $(System.PullRequest.SourceBranch) $(Build.SourceVersionMessage)'
  1. variables: We set a template file to get the environment variables.
  2. parameters: The parameters to be passed to the job template
    1. netlifySiteId: The site ID from environment variables.
    2. netlifyToken: The token from environment variables.
    3. message: The message is created using the Pull Request Number, Source branch and the comment of the commit.

Conclusion

By using Azure Pipelines, the Netlify limits are no longer a problem. We will see how we can add tests and run them against the deployed URL in a later post.