Consuming public cloud services locally with distributed clouds
Many clients want easily consumable public cloud like services while keeping data local e.g., due to compliance, security, or regulatory reasons
In this context, as another cloud deployment option, distributed cloud, has been coming up in recent client discussions. In this article I aim to introduce the distributed cloud topic, list the key benefits and highlight what needs to be considered when adopting distributed cloud technology.
Distributed clouds provide a means to extend public cloud services to multiple different locations while keeping the control plane centralized in a public cloud. A key aspect is that the data stays local and it’s possible to connect to other existing services at the same location with low latency.
Distributed cloud technologies are still in early phases with a small subset of public cloud services available on them – and with different levels of maturity. But they already have enough services to make them attractive to our clients. Distributed clouds enable simpler deployment and management of virtual machines and Kubernetes clusters. Management of many Kubernetes clusters is also challenging. Distributed clouds solve part of that complexity.
Having adopted public cloud PaaS services, distributed clouds enable for example deploying managed databases to on-premises locations simplifying application development and management tasks. Distributed cloud can provide for example high-availability, back-ups, and updates for the managed databases services. Additionally, one can leverage existing role-based access control, security policies and management tooling from a public cloud.
Use cases for distributed clouds
Use cases for distributed clouds include the above-mentioned regulatory reasons, where for example personal information (PI) cannot be stored or moved outside country borders or needs to be kept on-premises. Healthcare industry is a common example for such requirements. Even if there exists a public cloud region in the same country, its usage may not be allowed as data may theoretically be accessed by the public cloud provider. With a distributed cloud, one could control that the data stays local for example in a hospital basement while being able to leverage innovative public cloud services.
Another use case could be a factory where terabytes of IoT analytics data is needed to be processed locally. Moving such volumes of data to a remote public cloud location rarely makes sense, and data processing may be related to a production process requiring low latency. Again, a distributed cloud could provide added value by enabling to build modern applications leveraging public cloud data, ML and AI services locally.
A third example could be application modernization. There are many existing applications that are complex with multiple different components running on different virtual machines, Unix systems and leveraging large amounts of data in different databases. Moving such applications to a public cloud is very challenging whereas modernization of the applications in one go is not an option either. Distributed clouds provide a new option where a modern container platform complemented by key public cloud services can be deployed next to the existing infrastructure. This enables the application to be modernized gradually piece by piece. Once the most challenging application components have been modernized, the application could be migrated to a public cloud.
Building distributed clouds
All main public cloud vendors have their own distributed cloud platforms, including:
Distributed cloud implementations and deployment options differ from vendor to vendor. They may be deployed on bare metal, on top of a virtualization layer, to an existing Kubernetes cluster – in on-premises, edge or other public clouds. As with cloud in general, networking is an important aspect with distributed clouds. These are areas where there’s quite a lot of complexity. I.e., getting the technology stack ready to start consuming distributed cloud services is far from plug-and-play, and requires a lot of skills and effort. While some turn-key options with hardware included exist, one often needs to maintain the underlying infrastructure separately.
Above there’s a simplified and incomplete illustration of options related to a distributed cloud stack. Layers below the distributed cloud platform and interworking to other services need to be carefully considered.
It's relatively easy to setup a distributed cloud to another vendor’s public cloud, and then the management of the underlying infrastructure would be mostly taken care by the public cloud provider. But the benefits of this approach are not obvious – one basically gets a small subset of services from a public cloud running in a region of another public cloud. There would be a lot more native services to consume in this other public cloud, probably at a lower cost. This approach might work for example in case the selected cloud vendor does not have a local region available, but in many cases, a multicloud deployment model might be a more viable option.
In an on-premises environment things get much harder. One needs data center facilities, compute, storage and networking hardware, possibly a virtualization layer and possibly a container platform too. Maybe there’s an existing capacity service that can be leveraged, or one could buy the whole stack as an appliance from some vendors – but that often comes with a high price. Google Distributed Cloud Edge is an example of an all-in-one solution that comes with its own appliance hardware. Even ready-made packages lack the networking part which is one of the most complex parts. For example, how to securely connect the distributed cloud platform to a public cloud and how to connect it to existing network?
Recommended by LinkedIn
When planning for distributed clouds, a few example questions to answer before rushing into the implementation phase:
Having a distributed cloud at an on-premises location is not a silver bullet either. As the services of the distributed cloud platforms are still quite limited, one usually needs also other, existing, IT platforms in addition to the distributed cloud. So, in the end the complexity of an on-premises IT infrastructure will grow when introducing a new distributed cloud platform. But application development becomes simpler and some operational tasks like database management may be reduced.
Air-gapped data centers and Kubernetes multi-cluster management
In case of stricter regulation, and especially with air-gapped infrastructure in which the network is not connected to the Internet, the distributed clouds are typically not an option because a centralized control plane to a public cloud is not allowed.
There does exist, for example, Google Distributed Cloud Hosted for this purpose, but the scale required is too high for most clients. In such cases one needs to use do-it-yourself private clouds with on-premises container platforms instead (e.g., Red Hat OpenShift or VMware Tanzu Kubernetes Grid), optionally with multi-cluster management.
Kubernetes multi-cluster management is similar to distributed cloud by enabling creation and management of multiple clusters and application deployment and disaster recovery into those. However, with for example Red Hat Advanced Cluster Management (ACM), the control plane is basically on-premises, not provided by a public cloud (although the OpenShift cluster for ACM could be located in a public cloud too). More importantly, the key difference to distributed clouds is that one does not get the public cloud services to on-premises with that. Another example of multi-cluster management is VMware Tanzu Mission Control.
To add further complexity, one could use both multi-cluster management and distributed clouds together by managing Kubernetes clusters with for example ACM while deploying a distributed cloud platform in the clusters.
Conclusion
Distributed cloud is an interesting new technology and there’s a lot of interest to them now. The number of services supported by distributed cloud platforms is still limited but will surely grow going forward.
Some predict that in the future distributed clouds will enable running most public cloud services anywhere, for example in IoT devices or Wi-Fi access points - with a press of button. However, looking at how long time it currently takes to get public cloud services available in a new public cloud region, this kind of vision will take years or decades to become a reality. I find an evolution with vendors slowly adding new services to distributed cloud platforms more realistic. Distributed cloud technologies will not replace existing IT platforms but will complement them.
Another aspect to keep in mind is that like all technology, whether based on open source or not, a distributed cloud platform comes with a certain level of vendor lock-in. But if a specific public cloud provider has already been chosen, this is probably not an issue with distributed clouds.
Summary of the common benefits provided by distributed clouds:
While the benefits are clear, currently the challenging part is to design and implement the distributed clouds to on-premises data centers as it’s far away from plug-and-play. As with all IT, distributed clouds also require someone to manage and maintain it.
Kyndryl can help with distributed clouds. We can design, implement, and manage everything required to get a distributed cloud up and running - and we can provide this with a cloud like pay-as-you-go consumption model. We can also connect and integrate distributed clouds to your existing IT infrastructure. With our services, clients can focus on business outcomes and innovation, rather than building and maintaining IT infrastructure.