Auto ML

A couple months ago I started a new and exciting adventure of learning Data Science & Machine Learning, while I was wondering in this world of new notions And discovering interesting use cases of these technologies that could truly make the world a better place and that's what lead me to Deep learning, but before I started learning it and as you Already might know that Machine Learning is nothing but automating tasks and act smartly than the human brain and training itself and executing the tasks in a very accurate manner for real-world problems, during my learning process I came up with some questions on ML such as: 

how can you improve the efficiency of finding optimal solutions?

do you as a data scientist have to do all machine learning workflow for every problem you work on?

All those questions guided me to know that deep learning has progressed very rapidly over the past few years and with every week passing ,there are amazing new researches and discoveries published. There are many factors that are driving this rapid growth and expansion and one particular area of research that has been particularly interesting to watch is the proliferation of automated machine learning tools, starting from neural architecture search and now expanding to other parts of the machine learning workflow that can also be automated.

What is Auto ML ?

What exactly in machine learning can be automated ?

How does Auto ML exactly works ?

What are the benefits of Auto ML?

What is Auto ML ?

Auto ML stands on automated machine learning and it is the effort to fully automate end to end the machine learning process, this means in practice try to build systems where you just throw your data in , then by one click you get results (predictions), as you declare what you want and the system knows how to compute it.

What exactly in machine learning can be automated ?

let's talk about the traditional ML workflow,

as you can see, there are so many steps involved in it, as you see in the picture above you have to explore your datasets first, do some data exploration to get some insights then you have to clean your data and prepare it for processing, the next step would be feature engineering and identifying important features and drooping not so important ones, once you are done with all these steps, you have to pick different Algorithms and train your model on your training datasets, you choose your model based on it's accuracy and try to increase it by tweaking the model's parameters, then and only then your model is ready to deployment.

what if there is a way to skip all these intermediaries steps?

That's where Auto ML can help,

with the Auto ML, you just have to worry about one thing that is ingesting your data and the rest will be taken care of by Auto ML.

How does Auto ML exactly works ?

Starting from data preparation and ingestion: when Data first arrives, Auto ML can try to automatically detect the type of data in each column (Boolean, text, numbers...), it can even go as far as to try to detect the intent of a column, perhaps identify the target column or figure out if a column should be numerical, categorical, or just free text, then there's automated task detection which figures out whether to use a binary classification, regression, clustering, ranking or something else altogether. Once the data has been loaded in , the next consideration is feature engineering (the process of using domain knowledge to extract features (characteristics, properties, attributes) from raw data), it can be tedious and time consuming but automating some of this can save you a lot of time and even sometimes it can help expose blind spots since the computer will go through tasks including things like feature selection (the process where you automatically or manually select those features which contribute most to your prediction variable), pre-processing and extraction as well as the detection of skewed data or missing values. Next up is model selection while the model to use for a given data set and tasks may seem obvious to you, so building a computer system to automate this process for a variety of different data types is not quite so simple, model selection not only finds the general type of model to use but it can also do the architecture search to find the specific structure most suitable for a given data task. Finally there's the automation of the evaluation step everything from validation procedures to checking for mistakes as well as analysis and visualization of the results.

What Are the Benefits of Auto ML?

Leveraging Auto ML solutions aims to do the following options:

• Cost reductions
• Increased productivity for data scientists
• Democratization of machine learning reduces demand for data scientists
• Increased revenues and customer satisfaction
• Rolling out more models with increased accuracy can improve other, less tangible business results as well. For example, models lead to automation which improves employee engagement allowing them to focus on more interesting tasks.