Painless Delivery with Google App Engine & GitHub Actions
World of DevOps is a twisted one, especially when it comes to delivery. Between the endless solutions to choose from and picking "The One" that meets your need while keeping provisions for future scopes to settle in, one can get lost in the complexity as the product grows & diverges.
In this post, I'm going to dive in to Google App Engine but going to illustrate how it abstracts and thereby enhances a delivery pipeline for a simple app.
Assumptions
To imbibe this "walk in the park" article, understanding of the following are needed:
The Problem
Manually Provisioning Infrastructure
An monolithic architecture commonly looks like:
For simplicity let's only talk about the Compute Layer, where the code base executes and gets the job done. To keep up with the traffic, this layer should scale out automatically. The compute layer usually is a combination of multiple components like: Load Balancers, Auto Scaling Groups, Instances, Launch Templates, etc. The blue dotted container in the following diagrams shows what a compute layer looks like:
Multiple Functional Versions (Tenant Specific Customization)
Now, let's factor in a special requirement to give this article a greater value. The product to be deployed has Multiple Functional Versions, each dedicated to a particular tenant. In simpler words, the product behaves differently at some given feature points to suit the customized needs of different tenants. This is achieved by keeping a dedicated branch for each tenant (for example, org-1, org-2, and org-3) which looks something like:
In this scenario, we'll have to separately provision the whole infrastructure for each tenant before we could serve them. Keep in mind, they might need a staging or UAT environment so the efforts double right away.
VM Template Change
Let's also embark on a new avenue of thought. What is the App needs an updated version of the software platform/engine it is running in? For example, an upgrade from NodeJS 14 to 16? Or for some reason, upgrade the OS. This requires a change in the Launch/Machine Template/Image and relaunching infrastructure for all tenant with new Template/Image.
The CI Tool
A CI Tool like Jenkins adds a layer of cost & complexity. The VM hosting the CI tool would have to be standalone. Any connectivity failure between Agents & Controller nodes have to be managed. And most importantly, with each new tenant, the CI Tool has to be aware of the newly added destination infrastructure.
The solution
The solution is multi folded. On one side the infrastructure automation will be handled, and on the other the CI/CD operations will be simplified.
Hello! Google App Engine
Google App Engine is a managed service to handle computation needs that provisions, scales out & load balances traffic among VMs. Remember the dotted blue container? It is literally transformed into:
To deploy an app to Google App Engine, we need to have a file app.yaml in the our project which specifies the configuration of the infrastructure:
We can specify the instance class on which we want to run our code, the max or minimum number of instances we want and much. The official documentation contains all information regarding the content of app.yaml.
One important thing to notice here is the "service" property. This is crucial because each functional version of our app will be launched under a different service name. The convention we've used is: server1-<ENVIRONMENT>-<TENANT_NAME_OR_ID>
The code base is a very simple NodeJS Express application that looks like:
Once we've placed the app.yaml in our project, we run the command:
gcloud app deploy ./app.yaml
The logs looks like a successful deployment. Looking at the Google App Engine console:
Recommended by LinkedIn
We have the service. Now, trying to browse the service (either by clicking on the service name or by navigating to App's entry link given by gcloud CLI):
Hence, the App is running successfully. The portion "[main]" in the output text resembles the branch of repository which is in turn tenant & environment specific.
So our infrastructure is handled. It's time to handle the CI tool part.
Hey there, GitHub Actions
We'll use GitHub Actions to deploy our code to Google App Engine (which takes care of the infrastructure) to make our architecture look like as simple as:
Our aim is to trigger a GitHub action when a branch is updated. It's crucial to understand that we'll use the same YAML file to specify where the code will be deployed in each branch, but the branch name to pick the code from will be different.
The above image resembles the GitHub Workflow file for GitHub actions. As seen, the workflow is only set to be invoked via a change in branch "production-org-2". This workflow file actually resides in the branch "production-org-2" itself. So, each workflow file in each branch is set to be invoked via a change in that branch only. This is because deployment of the changed branches are desired and not any other. Through this implementation, only the branch that receives an update, will start it's deployment.
Down the memory lane (just 2 blocks ago), it can seen that in each branch the service name resembles the tenant and environment of deployment through app.yaml file. So this is how distinct deployment to Google App Engine is specified.
Let's Deploy!
We're only going to push to the repo, branch by branch. That's it. Everything will be taken care of from there by GitHub Actions & Google App Engine.
Pushing Branch production-org-1
GitHub Actions Deploys the updated app to Google App Engine for production-org-1
Pushing Branch production-org-2
GitHub Actions Deploys the updated app to Google App Engine for production-org-2
Pushing Branch production-org-3
GitHub Actions Deploys the updated app to Google App Engine for production-org-3
Hence, the deployment is complete. Let us now see if we have all the services in Google App Engine console:
Well, we do. Now let's check if all of them are accessible:
Hence we have simplified a complex delivery pipeline & infrastructure management scenario into a very basic solution.