Creating CI/CD pipeline using dynamic distributed clusters concept!

A cluster consists of one master machine and multiple worker machines or nodes. The master coordinates between all the nodes. A pod is the smallest unit of a cluster.

If we use Static Clusters then we already configure our VMs according to our requirements and they keep on running and thus utilizing all the major resources even if we are not using them so to solve this issue we use the concept of Dynamic Clusters.

Dynamic Clusters create the clusters only when we require , then they run the jobs and once the jobs are done they terminate the os .Thus preventing the over utilization of the resources.

Task Overview

1. Create a container image that has Linux and other basic configuration required to run Slave for Jenkins. ( example here we require kubectl to be configured )

2. When we launch the job it should automatically starts job on slave based on the label provided for dynamic approach.

3. Create a job chain of job1 & job2 using build pipeline plugin in Jenkins 

4. Job1 : Pull the Github repo automatically when some developers push repo to Github and perform the following operations as:

4.1 Create the new image dynamically for the application and copy the application code into that corresponding docker image

4.2 Push that image to the docker hub (Public repository) 

 ( Github code contain the application code and Dockerfile to create a new image )

5. Job2 ( Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command): Launch the application on the top of Kubernetes cluster performing following operations:

5.1 If launching first time then create a deployment of the pod using the image created in the previous job. Else if deployment already exists then do rollout of the existing pod making zero downtime for the user.

5.2  If Application created first time, then Expose the application. Else don’t expose it.

Commands used inside the docker file are shown below.

After launching the job it should automatically start the job on the slave os based on the labels , so for that we need to configure the jenkins and install 2 plugins Docker and Yet Another Docker.

Configuring Job 1

Configuring Job 2

 Job2 Should be run on the dynamic slave of Jenkins configured with Kubernetes kubectl command

Launch the application on the top of Kubernetes cluster performing following operations:

Job 1 output

Job 2 Output

Output Of The Kubernetes

The YAML file that is used for launching the pods (config file)

Build Pipeline View of Jobs