MLOPS - Automating Hyper Parameter Tuning of CNN Model using DevOps Tools
AIM of Project
We have to create a Container Image using Docker that has Python3, Jupyter-Notebook, Keras, TensorFlow, OpenCV, Numpy and Pandas Library Installed using Dockerfile. We have to Automate this Container in such a way that as soon as we Launch this Container in REDHAT Linux OS, it should Automatically starts Training the Machine Learning Model, which we have Created on Convolutional Neural Network using a DataSet Created by us. This Model consists of set of Hyper Parameters which we have to tweak every time when ever model is trained until it gives an Accuracy over 80%. Now, using Jenkins Automation Tool in RHEL we have to create 5 jobs which will Complete the given real life problem Automatically.
GitHub link - At bottom of the Page.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
What is Hyper Parameters?
In Machine Learning, a Hyperparameter is a parameter whose value is set before the learning process begins. By contrast, the values of other parameters are derived via training. It can be classified as model hyperparameters, that cannot be inferred while Fitting the dataset to Machine because they refer to the model selection task, or algorithm hyperparameters, that in principle have no influence on the performance of the model but affect the speed and quality of the learning process. An example of a model hyperparameter is the topology and size of a neural network. Examples of algorithm hyper Parameters are learning rate and mini-batch-size.
Different model training algorithms require different hyperparameters, some simple algorithms require none. Given these hyperparameters, the training algorithm learns the parameters from the data. For instance, Lasso is an algorithm that adds a regularization hyperparameter to Linear regression, which has to be set before estimating the parameters through the training algorithm.
Here, Hyper Parameters which we are twerking and are used in our CNN model.
In the above code there is Two Convolution Layer and Two Pooling Layer and the values which we are passing in the Convolution2D and MaxPooling2D like filters, kernel_size are one of the examples of Hyper Parameters we have to change these values by ourselves if our accuracy is not good. So we have to change the number of Convolution Layers and Pooling Layers in our model to get better accuracy.
In the above Picture there also number of epochs, steps_per_epoch are the examples of Hyper Parameters which we are going to tweark.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Convolutional Neural Network of this Model
What is Convolutional Neural Network?
A convolutional neural network (CNN) is a type of ANN used in image recognise and processing that is specifically designed to process pixel data. CNNs are powerful image processing, artificial intelligence that use deep learning to perform both generative and descriptive tasks, often using machine vision that includes image and video recognition, along with recommender systems and natural language processing.
Below is the architecture of my CNN Model before Hyper Parameter Tuning using DevOps
After making this architecture and we have to train our model by providing it all the training images in my case I have the images of My Face and one Family Members Face as model training data. The process of fitting or training our model is done by the Docker Environment. Now from here, I will work to mention the work of DevOps tools in this project because writing the code and training our model is not a big deal for anybody. The following points are explaining where Jenkins, Docker, Git is used for Hyper Parameter Tuning for Better Accuracy in this project.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Use of Git and Git-Hub
Git is mainly used for Uploading and Downloading Changes on the Developer Side and on the Automation Side by simple git push, pull, commit commands. Git will upload and download code from Github at the same time Github will use the feature called WebHook.
For notifying the Jenkins Job to download code automatically for performing further operations. For some idea behind Git and Github for webhook.
Using Git bash in my PC, i have uploaded code files on GitHub.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Docker Container
Docker is famous in the market for its Agility, that it can launch OS in just 1 to 2 Seconds. Docker is generally using Containerization technology. In this Project, Docker is used for launching Environments or OS for running Deep Learning or Traditional Machine Learning Code in it to get the best accuracy for the ML Model.
I've created One docker images of Deep Learning Packages for running my CNN Python Code in it.
To create an image using this Dockerfile just use the following command:
docker build -t "ImageName" "location of directory which contains Dockerfile"
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Automation - Jenkins
Jenkins is a free and open source automation server. It helps automate the parts of software development related to building, testing, and deploying, facilitating continuous integration and continuous delivery. It is a server-based system that runs in servlet containers such as Apache Server.
Jenkins Job - 1
Machine Learning Code(CNN): This job is Defined in such a way that it will automatically pull data from Github and will store data in the workspace of Jenkins as soon as the developer pushes the updated code on the Github. I have also used the Build Pipeline in this Automation which automatically start the second job as soon as First job is Completed.
As i have used Webhooks for Automation of upload of code from local pc to github as soon as i update code.
Jenkins Job - 2
Container Creating Job: This job is Developed in such a way that it will determine the type of packages used in the main file of python. As the packages used in the files initiate, the Automation tool Jenkins will launch that container which is having all the Deep Learning Packages of python installed in it such as Tensorflow, Keras, Numpy, Sklearn etc.
As we know all jobs are part of the build Pipeline, so as peer the requirement of the pipeline all the jobs have to be in series, which says one job after completion of another, so for this reason i have made this job run after project_job1 will have a stable build.
As for the conformation i am using the post build option which will inform me through a mail whenever there is a change in the project_job2.
Jenkins Job - 3
Model Training Job: This job is trained in such a way that it will Start the Training Process of our Deep Learning Model or CNN Model in the respective container which was launched by the Previous Job 2.
As we know all jobs are part of the build Pipeline, so as peer the requirement of the pipeline all the jobs have to be in series, which says one job after completion of another, so for this reason i have made this job run after project_job2 will have a stable build.
Jenkins Job - 4
CNN Model Tuning Job: This job has that ability that will determine the accuracy of the model trained in the previous step and check if the accuracy is less than 80% then it will automatically change Hyper Parameters in the Python File and again Push it on the Github so that this cycle from Job1 again starts. If the accuracy of the model is > 80% then it will notify the developer and will not change the Hyper Parameter. This job will run a Python script that will check the accuracy and do Hyper Parameter Tuning for this Model.
As we know all jobs are part of the build Pipeline, so as peer the requirement of the pipeline all the jobs have to be in series, which says one job after completion of another, so for this reason i have made this job run after project_job3 will have a stable build.
As we can see that it'll run the find_accuracy file in the docker to find the accuracy and if accuracy is less than 80%, it'll automatically tweak the Deep Learning Model and we just have to push it to the GitHub from Linux.
All the related files can be found on my Git-Hub repository (Link Given at bottom of the Page). These files are very important for this project, please refer to my GitHub Account to get the logic behind it.
Jenkins Job - 5
Project Monitoring Job: This job is specialized in such a way that it will run on a scheduled basis to check whether the containers are running fine or not, if my TensorflowOS container goes down then it will again launch that container. This job will start after the Job 2 and it check the container every minute.
As we know all jobs are part of the build Pipeline, so as peer the requirement of the pipeline all the jobs have to be in series, which says one job after completion of another, so for this reason i have made this job run after project_job2 will have a stable build for this i have also use Poll SCM option which will lead to check of the Project after every Minute.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Where we Reached?
So we reached to the end of this one Great Project where we used Integration two of the Best Technologies Available in Market i.e; Machine Learning and DEVOPS. Integration of these two technologies is known as MLOPS.
In this Project we have successfully completed our Aim of Tweaking the Hyper Parameters of an CNN ML Model using the Devops tools, in very Efficient way considering Time and Space Complexity. For more Details you can reach out my GitHub Repository!
