R, Python and Matlab® in IIOT

There is a lot of development in IOT (Internet of things) and especially in the development of message broker. If you search for IOT message broker you will find companies that are able to connect over a million devices by creating large clusters. They support different message formats such as AMQP and MQTT, where the latter is widely used in the connected world due to its small message size.

In the IIOT (Industrial Internet of things) world there is also a lot of new developments and there are some interesting posts on LinkedIn about local or on premise IIOT installations. The concept makes a lot of sense, since these solutions can easily be scaled up and migrated to cloud services later. Even on a small scale local IIOT installation you are using all the basic components, protocols and best practices of a much larger solution.

One of the main advantage is that IOT works great as an abstraction layer. You can swap out the message broker, the database or any of the micro services without any major changes to the architecture. The following from https://inductiveautomation.com/solutions/iiot shows this nicely:

The missing piece from the picture is streaming analytics. You can subscribe to IIOT data using scripting languages such as R, Python or Matlab® and apply sophisticated libraries, but it is hard to productize these solutions. The main reasons are that scripted solutions are slow, they don't integrate and scale well. The IIOT message protocols also lack the structure to support advanced analytics. This is of course by design, since for example MQTT was developed for a small message size to support low bandwidth connection.

To integrate R, Python or Matlab® scripts requires first to provide a structured payload in form of a JSON. The JSON needs to include the source and target topics in order for the micro service to subscribe to the source data and route them to the right destination. The other components are the calculation instructions. Micro services should be stateless, which means that each calculation has to start and complete in just one call. A simple design would need to include a series of

• push operations to send data to the calculation engine
• the execution
• and pull operation to retrieve one or more results

To perform advanced analytics requires to not only support single values, but also vector and matrix formats. This is a bit of stretch for the original intent of MQTT, but makes sense in IIOT to connect different micro services.

Once the message format if well formatted, the R, Python or Matlab® micro services will go through the exact same message flow. The following shows an example of a Matlab® IIOT micro service call:

The left side shows the JSON encoded calculation instructions (vector dot product) and the right side the returned result JSON.

In summary, advanced analytic integrate well in IIOT. Data can be routed to calculation micro services and the results consumed by any other IIOT component. Load balancing can be simply achieved by adding micro services and subscribing to the same topic.