DevOpsAL Task-6 Continuous Integeration and Delivery with Groovy approach.

This is the overview of task.

Perform third task with the help of Jenkins coding file ( called as jenkinsfile approach ) and perform the with following phases:

1. Create container image that’s has Jenkins installed using dockerfile Or You can use the Jenkins Server on RHEL 8/7

2. When we launch this image, it should automatically starts Jenkins service in the container.

3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins 

4. Job2 ( Seed Job ) : Pull the Github repo automatically when some developers push repo to Github.

5. Further on jobs should be pipeline using written code using Groovy language by the developer

6. Job1 :  

  1. By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )

  2. Expose your pod so that testing team could perform the testing on the pod

  3. Make the data to remain persistent using PVC ( If server collects some data like logs, other user information )

7. Job3 : Test your app if it is working or not.

8. Job4 : if app is not working , then send email to developer with error messages and redeploy the application after code is being edited by the developer

Prerequisites:-

-One any virtual box

-Inside it kubernetes minikube cluster setup

-Account in github.

Overview

I have made a system for continuous integeration and delivery with groovy approach means everything is handled by developer only. He is writing devloping code and doing the coding required jobs for also.Everything is managed by kubernets for contianers. It is monitoring pods means if pod failed it launch again.

Tools and Technologies

Docker , Kubetnetes, Github, Jenkins

Now lets start with the explanation part

> This is dockerfile i have used for launching jenkins inside a pod or container.

-inside this image i have configure jenkins and kubeclt for Kubernetes minikube cluster so that we can access cluster from the container.

FROM  centos
USER   root
RUN  yum update  -y
RUN  yum  install  wget  git curl python36  java  initscripts  yum-utils  -y
RUN  wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
RUN  rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
RUN  echo -e "jenkins  ALL=(ALL)  NOPASSWD: ALL" >> /etc/sudoers
RUN  echo -e "docker  ALL=(ALL)  NOPASSWD: ALL" >>  /etc/sudoers


WORKDIR /vish/jenkins
RUN   curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
RUN  yum install  jenkins  -y
RUN chmod  +x kubectl
RUN  cp kubectl  /usr/bin
COPY  ca.crt   /root/
COPY  client.crt   /root/
COPY  client.key   /root/
COPY  config   /root/.kube/config
EXPOSE  8080




CMD  java   -jar  /usr/lib/jenkins/jenkins.war

-This is the Kubectl configuration file for K8s minikube cluster.

apiVersion: v1
kind: Config


clusters:
-  cluster:
     server: https://192.168.99.101:8443
     certificate-authority:  /root/ca.crt
   name:  vishcluster


contexts:
-  context:
     cluster:  vishcluster
     user:  vishal
   name:  "myvish"
current-context:  "myvish"


users:
-  name:  vishal
   user:
     client-key: /root/client.key
     client-certificate:  /root/client.crt

Now i build above image and push it into docker hub repository using following commands.

docker build  -t   docker_hub_usrname/image_name

docker login   ## for login into docker repository
 
docker push   docker_hub_usrname/image_name

After this you have to do initial aunthetication for jenkins.

kubectl exec  pod-name cat /root/.jenkins/secrets/initialAdminPassword  ## it will give initial password of jenkins

-See jenkins is ready for use

-This is the .yml file for the jenkins with all the service, pvc deployment setup.

apiVersion: v1
kind:  Service
metadata:
    name:  jen-node-service
spec:
    selector:  
        env:    master
    type:  NodePort
    ports:
        -   nodePort:  31999
            port:  8080
            targetPort:  8080


---
apiVersion:  v1
kind:   PersistentVolumeClaim
metadata:
    name:  vish-jenkins-pvc
    labels:
        name:   vish-jenkins-pvc
spec:
        accessModes:
             -    ReadWriteOnce
        resources:
               requests:
                     storage:    1Gi


---
apiVersion:  v1
kind:   PersistentVolumeClaim
metadata:
    name:  vish-project-pvc
    labels:
        name:   vish-project-pvc
spec:
    accessModes:
        -    ReadWriteOnce
    resources:
        requests:
                storage:    1Gi


---
apiVersion:  apps/v1
kind:  Deployment
metadata:
    name:    vish-jenkins-master-deploy
spec:
    replicas:  1
    selector:
        matchLabels:
            env:    master
    template:
            metadata:
                name:   jenkins-pod1
                labels:
                    env:   master
            spec:  
                containers:
                -   name:   jenkins-container
                    image:   vish1234/jenkinsdsl:j1
                    volumeMounts:
                        -   name:   project
                            mountPath:  /vish/jenkins
                        -   name:   vish
                            mountPath:   /root/.jenkins
                volumes:
                -   name:   project
                    pesistenVolumeClaim:
                            claimName:  vish-project-pvc
                -   name:   vish
                    pesistenVolumeClaim:

                             claimName:  vish-jenkins-pvc

-Using following command you have to launch this set up. I have written evruthing in one single file but you can create different files for pvc(persistent volume claim) and service for providing outside connectivity with load balancing.

kubectl  create  -f  file_name.yml

-See jenkins pod is ready

To proceed further first you have to install some plugins for this setup in jenkins.You have to install Git and Github, build pipeline,job DSL and email plugins.

 To download this plugins go to Jenkins➝Manage Jenkins➝Manage plugins➝ Available and search for the required plugins.

->After this you have to create a seed job.

For this write a code in Groovy language and push it on GitHub.

This code contains the details about the jobs that are needed to deploy web-server.

.For creating Job_1

job("ak1_groovy"){
	   description("My job1")
	  scm{
	     github('Vishal-exe123/devopsalTask6' , 'master')
	}
	steps{
	  shell('sudo  cp  -rvf  *  /vish/jenkins')
	}
	triggers{
	   gitHubPushTrigger()
	}

}

.For job_2

job("ak2_groovy"){
	  steps{
	   shell('''
	     if   sudo  ls  /vish/jenkins  |  grep  html
	    then    
	             if  sudo kubectl  get pods --selector  "app in (apache)"  |  grep  apache-pod
	             then
	                   POD1=$(sudo  kubectl get pods  -l app=apache  -o jsonpath="{.items[0].metadata.name}")
	                   echo  $POD1
	                  sudo   kubectl  cp   /vish/jenkins/index.html  $POD1:/var/www/html
	                  sudo   kubectl  cp   /vish/jenkins/vish.html  $POD1:/var/www/html
	          else
	                  sudo  kubectl   apply  -f  /vish/jenkins/apachepod.yml
	                   POD1=$(sudo  kubectl get pods  -l app=apache  -o jsonpath="{.items[0].metadata.name}")
	                   echo  $POD1
	                  sudo   kubectl  cp   /vish/jenkins/index.html  $POD1:/var/www/html
	                  sudo   kubectl  cp   /vish/jenkins/vish.html  $POD1:/var/www/html
	         fi
	    else  
	          echo  "no html file" 
	    fi
	
	    if   sudo  ls  /vish/jenkins  |  grep  php
	    then    
	             if  sudo kubectl  get pods --selector  "app in (php)"  |  grep  php-pod
	             then
	                   POD2=$(sudo  kubectl get pods  -l app=php  -o jsonpath="{.items[0].metadata.name}")
	                   echo  $POD2
	                  sudo   kubectl  cp   /vish/jenkins/vishal.php  $POD2:/var/www/html
	          else
	                  sudo  kubectl   apply  -f  /vish/jenkins/phpod.yml
	                   POD2=$(sudo  kubectl get pods  -l app=php  -o jsonpath="{.items[0].metadata.name}")
	                   echo  $POD2
	                  sudo   kubectl  cp   /vish/jenkins/vishal.php  $POD2:/var/www/html
	         fi
	    else  
	          echo  "no html file" 
	    fi
	''')
	}
	  triggers{
	    upstream('ak1_groovy' , 'SUCCESS')
	}

	}
}

For Job_3

job("ak3_groovy"){
	   steps{
	         shell('''
	          status=$(curl  -o  /dev/null  -s  -w  "%{http_code}"   http://192.168.99.101:31000)
	          if  [[ $status ==  200 ]]
	          then
	               echo  "apache html is running"
	               exit  0
	         else
	              exit 1
	         fi
	
	         status=$(curl  -o  /dev/null  -s  -w  "%{http_code}"   http://192.168.99.101:32000)
	          if  [[ $status ==  200 ]]
	          then
	               echo  "apache php is running"
	               exit  0
	         else
	              exit 1
	         fi
	''')
	}
	  triggers{
	    upstream('ak2_groovy' , 'SUCCESS')
	}
	  publishers {
	    extendedEmail {
	        recipientList('vishalyadav831874@gmail.com')
	        defaultSubject('Job status')
	               attachBuildLog(attachBuildLog = true)
	        defaultContent('Status Report')
	        contentType('text/html')
	       triggers {
	            always {
	            subject('build Status')
	            content('Body')
	           sendTo{
	              developers()
	             recipientList()
	            }
	        }
	    }
	 }
	}

	}
}

Following are two deployment yml file for launching the two different pods according to file.Following codes are pushed to the GitHub repository from where Jenkins can fetch them and deploy the web-server.

.This is for html code

apiVersion: v1
	kind: Service
	metadata:
	  name: apache-service
	  labels:
	    app: apache
	spec:
	  ports:
	    - nodePort: 31000
	      port: 80
	      targetPort: 80
	  selector:
	    app: apache
	  type: LoadBalancer
	---
	apiVersion: v1
	kind: PersistentVolumeClaim
	metadata:
	  name: apache-pv-claim
	  labels:
	    app: apache
	spec:
	  accessModes:
	    - ReadWriteOnce
	  resources:
	    requests:
	      storage: 20Gi
	---
	apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
	kind: Deployment
	metadata:
	  name: apache-pod
	  labels:
	    app: apache
	spec:
	  selector:
	    matchLabels:
	      app: apache
	  strategy:
	    type: Recreate
	  template:
	    metadata:
	      labels:
	        app: apache
	    spec:
	      containers:
	      - image: vimal13/apache-webserver-php
	        name: apache
	        ports:
	        - containerPort: 31000
	          name: apache
	        volumeMounts:
	        - name: apache-persistent-storage
	          mountPath: /var/www/html
	      volumes:
	      - name: apache-persistent-storage 
	        persistentVolumeClaim:

	                              
               claimName: apache-pv-claim

.This is for php+html.You can do same for python, java or anyother webserver.

apiVersion: v1
	kind: Service
	metadata:
	  name: php-service
	  labels:
	    app: php
	spec:
	  ports:
	    - nodePort: 32000
	      port: 80
	      targetPort: 80
	  selector:
	    app: php
	  type: LoadBalancer
	---
	apiVersion: v1
	kind: PersistentVolumeClaim
	metadata:
	  name: php-pv-claim
	  labels:
	    app: php
	spec:
	  accessModes:
	    - ReadWriteOnce
	  resources:
	    requests:
	      storage: 20Gi
	---
	apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
	kind: Deployment
	metadata:
	  name: php-pod
	  labels:
	    app: php
	spec:
	  selector:
	    matchLabels:
	      app: php
	  strategy:
	    type: Recreate
	  template:
	    metadata:
	      labels:
	        app: php
	    spec:
	      containers:
	      - image: vimal13/apache-webserver-php
	        name: php
	        ports:
	        - containerPort: 32000
	          name: php
	        volumeMounts:
	        - name: php-persistent-storage
	          mountPath: /var/www/html
	      volumes:
	      - name: php-persistent-storage 
	        persistentVolumeClaim:

	                              
                   claimName: php-pv-claim

To create seed job: Pull the GitHub repository automatically when some developers push repository to GitHub

Output of Seed job

After execution of seed job, following jobs are created with their output.

.Job1

output of job1

Job2

output of job2

Job3

output of job3

All the jobs worked successfully with seed job.

Build Pipeline view.

Following simple web i have deployed.

My github url for groovy code.

https://github.com/Vishal-exe123/devopsGroovy

Github url for webserver and deployment code.

https://github.com/Vishal-exe123/devopsalTask6

Thank You.