MLOPS PROJECT

ML-Devops:

Mlops is where both Machine learning and operations based on the service and communication between the 2 sides happen. When paired with tools used for devops to produce more efficiency and convienience,we arrive the term ML devops

Dataset used and its purpose:

The dataset is taken from kaggle.com. This model helps to predict given cell whether it is parasitised or unaffected from maleria.

Objective:

1.Create 2 or 3 docker images using dockerfiles each having its own required libraries.

2. Push your python code to github and make jenkins o pull your specified workspace on your local machine

3.Ask jenkins to recognise type of code whether it is CNN or ANN code then push it to its respective docker container.

4.Make jenkins to fetch accuracy of the model trained and add crp layers and increment and decrement the values if accuracy is less than 90 percent.

5.If container fails then ask jenkins to make container pick up from where it left and push updated code to github

procedure:

1.Create Dockerfile for your CNN image

we then run this dockerfile building our docker image centcnn

$ docker build -t centcnn .

2.We will now integrate Jenkins and Github to pull our ML/CNN code

JOB1:Pull git hub repository contents automatically onto your local repository automatically i.e. from got hub to root/task/ using PollSCM

sudo cp * -v /root/task

this command copies the content from GitHub repository to your local workspace

JOB2:Checks whether the downloaded code belongs to CNN or ANN and then sends it to respective docker container to run the code.

Execute the following script:

if[ '$(sudo cat /root/task//maleria.py)|sudo grep Convolution2D']
then 
    if sudo docker ps -a | sudo grep cnn 
    then
        sudo docker rm cnn
    fi
    sudo docker run -t -v /root/task/:dock/--name cnn --rm centcnn /dock/maleria.py
else
    if sudo docker ps -a | sudo grep ann 
    then 
        sudo docker rm ann
    fi
    sudo docker run -t -v /root/task/:sklearn/--name ann centcnn:v1

exit 1
fi

this code checks if maleria.py contains Convolutio2D which means it belongs to CNN code, if yes it checks for existing container by name CNN,if yes,it removes it.

Then we mount our base directory to the container.

JOB3:Checks if the accuracy of trained model less than 80% and if it is various changes are initiated

we have to add Dense() layers and Convolution2D layers. Trial and error tweaks are executed.

if [$ sudo cat /root/task/Accuracy.txt)-gt 80]
then 
    echo "Accuracy Achieved $(sudo cat /root/task/Accuracy.txt)"
    exit 0
else
    sudo cd /root/task/
    sudo sed -i "18i model.add(Convolution2D(filters=32,kernel_size=(3,3),activation='relu',input_shape=(28,28,3)))"/root/task/maleria.py
     sudo sed -i "19i model.add(MaxPooling2D(pool_size=(2,2)))"/root/task/maleria.py
      sudo cd
      sudo cd /root/task
      sudo git init
      sudo git config --global user.email "chinmaypai657@gmail.com"
      sudo git config --global user.name user.name "Chinmay S Pai
      sudo git add
      sudo git commit . -m "Tweaked code"
fi

The first line opens Accuracy.txt file that contains model accuracy and if conditions check if accuracy is less than 80%

NOTE:We access the file from local directory rather then the doker container because earlier we mounted this directory,in simple the content on mounted directories are synced between local and container directory

else we uses to add more convolution layers to increase accuracy.Then we initialize git repositoryand github repo is updated again and PollSCM triggers again and job 2 run again with modified code.

JOB4:The purpose is to send email after build is sucessfull.

Pluggins Required:Email and Email extended pluggin and mailer pluggin required.

JOB5:This job checks the status of the running container and re initiates the failed container

if sudo docker ps | grep cnn
then
    echo "CNN Container is Running Fine"
else
    sudo docker run -t -v /root/task/:/dock/name cnn --rm centcnn /dock/maleria.py
fi

NOTE:We havent monitored ANN container.Our main Objective is to complete the given task objective.

The model trained on this container takes very less no of images as its training and testing set due to lack of dedicated GPU

The machine specifications and software requirements for this mini projects:

specifications:intel i5 8th gen , 8GB RAM

software: VirtualBOX,RHEL-8,DOCKER,JENKINS

SPECIAL THANKS TO WORLD RECORD HOLDER MR.VIMAL DAGA SIR AND LINUX WORLD INFORMATICS FOR GIVING KNOWLEDGE AND OPPORTUNITY TO WORK ON THIS TASK.

TEAMMATE: Kevin D Goveas