Recognizing components with pytorch part II

In the first article I used a pretrained image dataset to train a number of different electronic components. I used a CNN architecture with feature extraction of the PyTorch model. In feature extraction, we start with a pretrained model and only update the final layer weights from which we derive predictions. It is called feature extraction because we use the pretrained CNN as a fixed feature-extractor, and only change the output layer. 

Deploy the model for production

If you install Python on a laptop and the library needed you can run it with good performance without any problem. It could be run on a Windows machine without a GPU but if you want to use the model in a mobile application or a web application then you need to deploy it as an API that you can POST images to and get a result back. The traditional aproach is to use Flask with Gunicorn and Nginx to serve the model from Amazon AWS, Azure, Google Cloud or similar cloud services. That could be difficult to setup, scale and optimize.

For this example I used a recently announced cloud service called Panini AI. After reading an article at Medium Towards data science about it I followed the example and deployed my model.

What is Panini?

From the Panini AI website you get the following explanation: Panini is a platform that serves ML/DL models at low latency and makes the ML model deployment to production from a few days to a few minutes. Once deployed in Panini’s server, it will provide you with an API key to infer the model. Panini query engine is developed in C++, which provides very low latency during model inference and Kubernetes cluster is being used to store the model so, it is scalable to multiple nodes. Panini also takes care of caching and batching inputs during model inference.

Panini is in Beta and are free for Beta testers if the model is below 2GB in size.

After uploading the model, main.py and requirements.txt to Panini. I tested the model by sending an image in json format with Postman and from python.

I tested with an image of an optical transceiver receiver SFP+ which it correctly recognized in 149 milliseconds.

Same test from a Jupyter Notebook.

Now we have a working and scalable API. To inlcude it in an application we also need to add authentication, then we can serve it to a mobile app or other frontends.