High-level overview of the Analytics/Data Science process

You will not learn unless you do it yourself.

Hey! Thanks for checking out my newsletter. I am grateful to you for taking the time out of your day to read this piece. This is my first article in my very first newsletter. I am going to state the purpose of this newsletter so that it reaches the right audience. These are my own views and opinions based on trials and errors, learning and experimentation.

• Topics: Need-to-know Natural Language Processing(NLP) and Machine Learning(ML) techniques/algorithms, model deployment process and the corresponding tools with practical examples, Machine Learning Operations(MLOps), building you personal portfolio(Resume, LinkedIn, GitHub etc.), practical stuff I wish I knew before starting a job in data science, providing a clarity on various job titles in this field.
• The above are broadly the topics that I will be covering in all my articles and will adapt the content based on feedback from readers. Negative criticism is appreciated (because: read the first quote of this article once more)

Without any more delay let us dive into the first topic which I have been reading about and implementing for the past year - MLOps: Bringing structure and organization to the Analytics Process

Do we need MLOps?

Thinking about a machine learning solution as a product has become very important in the recent months and for a good reason. I believe that one of the fundamental questions we should be asking ourselves as A.I developers is, “How can I effectively deliver a machine learning solution that benefits the customer?”. I specifically write “machine learning solution” and not “machine learning model”, reason being that by building a model you have completed 50% of the solution. The rest involves packaging your model as a product and delivering it to the customer. The following might not be the best analogy to clarify my previous statement however, think about the design and delivery of a car on a very high level. Building a machine learning model will be equivalent to designing the internal combustion engine along with everything that goes with it including the pistons, crankshaft, valves, fuel injectors etc. When it comes to selling the car to a potential buyer, do you deliver the engine directly to the customer and ask them to use it or do you package the engine, battery along with the car’s body, tires, steering wheel, and all other parts the customer needs to be able to start driving? This is what I believe to be the concept of MLOps: a philosophy of thinking that teaches us to deliver a machine learning solution as product to the customer. The rest of the article will delve into the mechanics of practically implementing the above ideas.

High-level overview of the Analytics process

I will be focusing on this topic to get started before we start dive deeper in later articles

Understanding the business problem, audience and requirements

• This is the most crucial step of the entire process as this is where it begins. Being able to understand the business requirements, data sources and whether A.I is even needed to solve the current problem are some of the key issues to be addressed at first.
• One might be thinking that this is pretty standard across the software industry. That is true however, when it comes to designing A.I solutions, a magic dust is blown right into our face and we start thinking about applying A.I before we even hear the actual problem and whether we truly need A.I in the current situation.
• Once we have a good understanding of the business requirements, we need to move on to next important task of scouting for data. A.I can be summarized in one single equation:

AI = Code + Data.

• From this equation one can see that we need to be able to have a good and reliable data source. Having a “reliable data source” is a dream state however, conducting multiple meetings and interviews with your business partners/stakeholders will sooner or later reveal the data sources that would not have been thought to be used to solve the problem
• The last thing is the right mindset and enthusiasm from your clients. They need to be just as excited as you(the developer) are in seeing the A.I solution come to its fruition. Your efforts should not go to waste if ultimately the business actions results in “archiving your git repository”. 
• Now we move on to the experimentation phase where Data Scientists start exploring the data, finding out any inconsistencies with the data(this is more toward Data Engineering as well which I will get to later), getting back to the business teams for further clarifications, building & testing various models and iterating these steps until we have a good model and effective model to answer the business problem

Now you have completed 50% of the task. The remaining 50% is all about thinking beyond the modelling phase. This would be the design phase that includes answering some of the following questions

• Where do I store my model? Is it going to be a batch or a real-time process?
• Where do I store my data and what is my data storage/ingestion solution?
• Do I need to use one of the cloud-based solutions or can it be done on-prem?
• How am I going to showcase my model’s results to the client? Do I create a website? How do I collect the input data from the user?
• Do I need to establish a feedback loop in order to retrain my ML model? How often do I need to retrain it?
• How do I deploy my solution? Do I need to wrap my code as an API that waits for client requests, or do I schedule my code to be run at some predetermined frequency?
• This design phase can either be conducted either before, after or in parallel to the modelling phase.

I could keep going with the questions; however, these design decisions also depend on a particular use case and the above questions are not the complete list. They are only meant to give you a broad idea.

• Once you have deployed your solution, while also getting feedback intermittently during the whole process from the business/clients you would go back and forth with your users to solve any more potential issues (this is going into the topic of performing various rigorous tests which will be discussed later)
• This briefly covers a very generic high-level overview of process involved in designing an A.I solution.

Coming up next:

• Typical DevOps process
• Code scanning, versioning, testing and quality checks
• Merging git branches and upgrading versions
• Integrating and deploying continuously(CI/CD)
• How can we combine ML and DevOps?

Once again, thanks for reading my article and I hope you learned something new today!