Reliable, Scalable Computing at the Edge for IoT – Containers and Orchestration
Cloud computing has greatly simplified the reliable deployment and scaling of applications. Public cloud infrastructure, as provided by AWS, Azure and Google, hide what is required to scale compute with workload and to keep programs running even as compute nodes go offline. These services are critical for making scalable, high availability IoT systems. How is that cloud-like capability replicated when moving from a public cloud to a private edge? This article will give a high level overview of some key ideas in this scenario.
There are three main ideas which provide the underpinning of scalable, highly available edge compute:
- Service Oriented Architecture (SOA)
- Containers
- Orchestration
TL;DR - SOA structures programs to be more easily scalable. Containers provide the mechanism by which programs can be more easily and repeatably deployed. Orchestration is the mechanism by which the "right" programs are started and stopped at the "right" times.
Service Oriented Architecture (SOA)
The traditional way to architect software is to make the code monolithic. This means that all the "things" a program needs to do are coordinated within that single program. The tasks are typically broken into functions (or methods in the case of object oriented programming) and those functions are called in a particular order based on the logic of the application. Schematically this can be shown as below:
Converting a monolithic program into a service oriented program involves breaking up the functions into services. Services are independent programs that understand when to perform some function and then performs that one function. For example, a service might be waiting for a message that some data has been generated. Then it might read that data, generate new data by running some algorithm, write the result and send a message that the new data is available. It is not as simple as putting each functional code block into a service, however. This idea can be represented as below:
This way of structuring an application is useful because it allows the program to scale more efficiently as compute loads increase. In the monolithic case, increased workload requires starting up more instances of the monolithic program (i.e. starting up more threads of the program assuming it was written to support multi-threading). This could be visualized like:
However, this may be a waste of resources if the increased workload falls mainly on one or two functions. This would leave relatively unused functions running, wasting system resources. Service Oriented Architecture avoids this problem by allowing the system to start up more of some services than of others. This would allow a heavy ingest load to be handled by starting more ingest services while leaving the number of alarm services unchanged. Schematically this might look like:
SOA provides a more efficient way to scale a program to deal with increased loads.
Containers
Programs require specific environments (operating system, libraries and environmental settings) to run properly. When only one program is running on a computer and the hardware is installed and maintained by the programmer, this isn't a problem. However, when multiple applications, each with its own environmental requirements, are run together on computers that are maintained by administrators managing many different systems, it is easy for a program to end up in the "wrong" environment or co-habitating with an application which has conflicting environmental requirements (e.g. the two programs require different versions of the same library). This results in errors that can difficult to understand and to address. Virtualization was created to help avoid this problem by putting each application into its own isolated space (a virtual machine). This allows the different programs to have their own operating system, libraries and settings. VSphere by VMware and Hyper V by Microsoft are examples of commonly used virtual machine platforms. However, virtualization is resource intensive: each o/s requires a significant amount of system resources to run and each o/s license can be expensive. Containerization was created to provide a lighter weight method of isolating applications from each other. Docker is a commonly used container platform. At a high level, virtualization and containerization can be thought of like this:
At a high level, the net results are similar but containers provide a more efficient way to manage programs.
Orchestration
Orchestration is how the right programs are started up, unneeded programs are shut down and programs are moved from failed servers to working servers. Docker Swarm by Docker Inc. and Kubernetes (open source) are two examples of commonly used orchestration platforms.
With monolithic programs, orchestration is largely "built-in" because all of the necessary functions are started together. With service based applications, the services can be managed in more complex ways. It is possible to define how services are related and how many of each service to run under what conditions. Related services can be defined in a pod:
Once defined, the number of each service can be scaled up and down depending on the system loading. When the amount data being ingested is low, the level of compute required is also low. Fewer service instances are needed to manage this load:
When the amount data being ingested is low, the level of compute required is also low. Fewer service instances are needed to manage this load. However, if the data loading increases, the amount of compute will also increase. The orchestration system might respond by increasing the number of services running like this:
Orchestration also allows for services which have failed for some reason to be shut down and replacements started up on the same or different server hardware within the cluster:
This allows the system to recover gracefully from limited hardware failures or some other aspect of the services which cause them to become non-responsive.
Orchestration is the technology which allows service based software, deployed in containers, to realize its potential and provide a cloud-like experience even at the edge.
