Automation of Machine learning With Devops

MLOPS--

mlops(a compound of “machine learning” and “operations”) is a practice for collaboration and communication between data scientists and operations professionals to help manage production ML (or deep learning) lifecycle. Similar to the DevOps or DataOps approaches, MLOps looks to increase automation and improve the quality of production ML while also focusing on business and regulatory requirements. While MLOps also started as a set of best practices, it is slowly evolving into an independent approach to ML lifecycle management. MLOps applies to the entire lifecycle - from integrating with model generation (software development lifecycle, continuous integration/continuous delivery), orchestration, and deployment, to health, diagnostics, governance, and business metrics.

Devops-

DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market.

Problem Overview :

1. Create docker container image that has python installed in it along with all the essential libraries required for training Machine learning model.
2. Create number of jobs in jenkins to test , notify , rebuild , tweak the machine learning model in order to get desired accuracy.

Lets first discuss the requirement of this project.

1. Docker image for running CNN & ANN program.
2. Docker image for running Regression & Classification program.
3. Jenkins & Docker & Git-GitHub

1. Build Docker images for TensorFlow and sklearn installed using Dockerfile:

2. Jobs in Jenkins:

job1) copy github code .

Whenever developer push any code in Github, this job automatically detect and copy in host OS.

• Job2(Deploy container for model_train):

If my job1 is successfully built, it triggers job2 and launches the container. By looking at the code or program file, this job automatically launch the respective docker container(either Keras os Sklearn).

Job 3: tweak If the model which is trained in Job2 has less accuracy than the developer desire, then this job will be executed which do some changes in the hyper-paremeters until the desired accuracy is achieved.

Thank You............