ML+DevOps (MLops) ==automated ML
why 60 percent of the Machine learning project is not implemented?? because most of the machine learning model requires automation and that why lots of the project of the ML model doesn't get implemented. So to automate the model we have to integrate the DevOps with ML.
here is some small example that could be helpful for you.!!
STEPS:
- we have to create a container image that has python3 and Keras or NumPy install using a docker file.
2. when we launch this image it will automatically launch a ready to use container image for machine learning model.
3. NOW we have to create JOB1, JOB2, JOB3, JOB4 and JOB5 using build pipeline plugins in Jenkins.
JOB 1
IT will pull the repo automatically when some developer pushes the repo to GitHub i.e. whenever the developer writes a code and ready to commit to GitHub then the job1 will be triggered automatically that will cp the code to your Workspace.
JOB 2
BY looking at the code or program file Jenkins will automatically start the respective machine learning software installed interpreter which is installed in the image that we created using docker file and start training the model .i.e. (if the code is in CNN it will automatically start the container that has already install all the software requires for the CNN processing )
JOB 3
This job will check the accuracy of the model. if the accuracy of the model is less then the threshold you have set then it will launch a python program that can tweak some HYPERPARAMETERS which is very crucial for machine leaning and will try to improve the accuracy of the model.
JOB 4
IT will retain the accuracy of the model and using mail it will inform the developer
JOB 5
It will act as a monitoring job that means if the container where app is running fails due to any reason then this job will automatically start the container again from where the last trained model left.
All these steps complete the Training of ML model or Deep learning model using the automation with the help of Jenkins.
ps: I have used Deep learning as an example in which I have used mnist dataset and trained the model using CNN.
this complete the whole pipeline
Code is provided in my GitHub repository
URL: https://github.com/vmukul41/MLops-task3.git
