Sharing my GitLab CI/CD tools, for App Engine Node Apps

When using GitLab to deploy our Node apps to App Engine, I found the tooling lacking. So I built my own tools and am now sharing them with you.

If you’re looking for a getting started with App Engine and GitLab CI /CD check out https://medium.com/google-cloud/automatically-deploy-to-google-app-engine-with-gitlab-ci-d1c7237cbe11.

GCloud Node Docker Image — for CI / CD Scripts

DockerHub: https://hub.docker.com/r/rlancer/gcloud-node

Use this image in your .gitlab-ci.yml, it pre-installs the following for you :

• Node
• NPM
• GCloud CLI
• gae-ayaml-env
• install-me-maybe

Usage

Inside your .gitlab-ci.yml set the Docker image to rlancer/gcloud-node:1.0.2

deploy:
  image: "rlancer/gcloud-node:1.0.2"
  script:
    - install-me-maybe # or npm i
    - npm run build
    - gae-ayaml-env
    - echo $GCLOUD_SERVICE > /tmp/$CI_PIPELINE_ID.json
    - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
    - gcloud --quiet --project $GCLOUD_PROJECT_ID app deploy app.yaml

App.yaml Environment Variable Generator — the easy way to manage environment variables

Npm: https://www.npmjs.com/package/gae-ayaml-env

AppEngine, for better or worse, manages Environment Variables in its application configuration file app.yaml. This can be problematic if you don’t want to have environment specific information in your source. Our solution for this is to use a templated version of app.yaml and then generate a new one.

Usage

Set your environment variables in GitLab (or other system), prefix variables you would like to persist in app.yaml with “APP_”, for example:

Create an app.template.yaml file include everything sans environment variables

runtime: nodejs10
env: standard
automatic_scaling:
  min_instances: 0
  max_instances: 2
service: default
env_variables:
  NODE_ENV: 'production'

In your CI / CD process run gae-ayaml-env to emit a populated app.yaml file, make sure you do not commit an actual app.yaml file as it will be overwritten.

Example for GitLab

deploy:
  image: 'rlancer/gcloud-node:LTS-229'
  script:
    - npm i
    - npm run build
    - npx gae-ayaml-env
    - echo $GCLOUD_SERVICE > /tmp/$CI_PIPELINE_ID.json
    - gcloud auth activate-service-account --key-file /tmp/$CI_PIPELINE_ID.json
    - gcloud --quiet --project $GCLOUD_PROJECT_ID app deploy app.yaml
  only:
    - prod

The system will write an app.yaml file fully populated with all the variables prefixed with “APP_”.

runtime: nodejs10
env: standard
env_variables:
  APIMARKET_FROM: '******************'
  DB_DATABASE: '******************'
  DB_HOST: '******************'
  DB_PASSWORD: '******************'
  DB_USER: '******************'
  NODE_ENV: production
  SLACK_APP_ID: '******************'
  SLACK_BOT_TOKEN: '******************'
  SLACK_CLIENT_ID: '******************'
  SLACK_CLIENT_SECRET: '******************'
  SLACK_OAUTH_REDIR: '******************'
  SLACK_SIGNING_SECRET: '******************'
  SLACK_TOKEN: '******************'
  SLACK_VERIFICATION_TOKEN: '******************'
automatic_scaling:
  max_instances: 2

Install Me Maybe — Idempotent NPM Installs

npm: https://www.npmjs.com/package/install-me-maybe

Prevents your Node modules from reinstalling if they are already installed. It will require you to enable cache on GitLab, certain runner types currently do not support caching, such as GKE based runners.

How it works

Creates a hash of your package-lock.json or yarn.lock and writes that hash to a file called .installmemaybe.

On the next call to install-me-maybe the contents of the hash file are compared with the new hash, only then will an install be triggered.

Installs your NPM modules only if they are out of date, designed for CI / CD systems.

Usage

Install it globally:

npm i -g install-me-maybe

After it’s installed just call install-me-maybe as many times as you’d like it will only reinstall if your lock file has been updated.