Openshift Virtualization & Multus - Chapter 1

Note: The baseline networking constraints discussed here apply to both standard Kubernetes and OpenShift. While we recommend OpenShift Virtualization because it includes the essential features to solve these problems out-of-the-box, the solutions we cover can also be achieved in any standard K8s environment.

I - Intro

Kubernetes solved many difficult problems, but not all of them.

One of the most evident appears when we try to take it outside its comfort zone: critical workloads with strict networking requirements.

In industrial, control, or telecommunications environments, the network is not just a communication medium — it is part of the system. There are traffic planes with very different behaviors, risks, and priorities. Control traffic must be predictable and low-latency; monitoring traffic requires visibility; management traffic requires secure and auditable access. Mixing them is not an option.

The good news is that we are not forced to choose between cloud-native and fine-grained networking control. Kubernetes and OpenShift provide the building blocks needed to close this gap, as long as we understand where to extend the base model.

In this article we will walk through that path:

• Starting with the default networking model for Pods.
• Understanding why it falls short in advanced scenarios.
• Introducing Multus CNI as a key component for multi-homing.
• And showing how OpenShift Virtualization allows integrating critical virtual machines without leaving the Kubernetes ecosystem.

If you have ever thought “this doesn’t fit in Kubernetes”, this article is probably for you.

II - The Scenario

Why 5G Breaks the Traditional Networking Model

Before putting on our engineering helmet and configuring interfaces, we need to understand the why. Why do we complicate things with multiple interfaces and advanced configurations?

The short answer is that the telecommunications world (and increasingly the industrial world, such as SCADA systems) does not operate like a conventional web application.

1. From VNF to CNF: A Strategic Evolution

A few years ago, the industry transitioned from physical Network Function appliances to Virtualized Network Functions (VNFs). It was a major step forward, but we were still dragging around heavy virtual machines.

Today we are experiencing the next revolution: the shift to CNFs (Cloud Native Network Functions).

This is not just a change in terminology. Moving from a monolithic VM to containerized microservices responds to both survival and business needs. According to Red Hat’s perspective (and the broader industry), this transition is driven by several critical factors:

• Operational Efficiency: CNFs deploy faster and consume fewer resources than traditional VNFs.
• Automation and DevOps: We need to be able to update a cell tower or a network core with the same ease as updating a mobile app.
• Continuous Innovation: 5G requires experimentation, failing fast, and deploying new features without bringing the network down.

2. The 5G Architecture: A Maze of Interfaces

This is the critical point for Telco architectures. When we look at the architecture of a 5G core, we do not see a simple server receiving HTTP requests. Instead, we see a complex mesh of functions such as AMF (Access and Mobility Management Function), SMF (Session Management Function), and UPF (User Plane Function). (link: https://www.3gpp.org/technologies/5g-system-overview)

Each of these “boxes” has very different communication requirements that cannot be mixed:

1. User Plane: This is where the gigabytes of user video and data flow through. It requires massive bandwidth and minimal latency.
2. Control Plane: This is the traffic that determines who connects and how. It is lightweight but critical; if a packet is lost here, the call drops.
3. Management Plane: (Often “hidden” in diagrams). This is the network through which administrators configure and monitor the equipment.

In a standard container architecture, all of this would be mixed through a single interface, eth0. In Telco environments, that is unacceptable from both a security and performance standpoint. We need to physically segregate these traffic flows.

3. Network Slicing and the Hardware Challenge

To make things even more complex (or more interesting), 5G introduces Network Slicing.

Imagine being able to divide the network into virtual slices, where each slice has different superpowers:

• A slice for Massive IoT: connects millions of sensors with minimal bandwidth.
• A slice for Ultra-Low Latency (URLLC): used for remote surgeries or autonomous vehicles.
• A slice for Mobile Broadband: enabling 4K streaming on the subway.

To achieve this, functions such as NG-RAN or AMF need privileged access to hardware. A slow virtual network is not enough; we need technologies such as SR-IOV so the container can communicate directly with the network card and guarantee the performance required by the 3GPP standard.

The Challenge:

We need a platform capable of orchestrating these containers (CNFs), understanding that a single function may require three different network interfaces, direct hardware access, and specific VLAN configurations, all managed automatically.

This is where plain Kubernetes falls short, and where the combination of OpenShift Virtualization + Multus becomes our Swiss army knife.

III - Pod Networking

To understand why we need advanced tools like Multus, we first need to understand how a normal Pod works inside OpenShift or Kubernetes.

Imagine a conventional Pod as a house with a single entrance and exit door.

In the world of traditional web microservices (an e-commerce site, an API, a blog), this works perfectly. Kubernetes is designed under the premise of simplicity:

1. The Pod is created.
2. The CNI (the cluster networking interface, which in OpenShift is usually OVN-Kubernetes) assigns it a single network interface, the well-known eth0.
3. That interface receives a single IP address within the cluster network.

All traffic flows in and out through it. If your application needs to talk to the database, it goes through eth0. If it needs to respond to a client, it goes through eth0. If it needs to send monitoring metrics, that also goes through eth0.

What happens when we bring this into the Telco world?

This is where the model breaks. Telecommunications applications (such as a VNF or CNF) are not simple web servers; they are complex systems that require strict order and separation.

In a typical Telco scenario, you have very different requirements that clash with each other:

• Control Plane: This is the brain. Light traffic, but critical and sensitive. Not a single packet can be lost.
• User Plane: This is the muscle. A massive torrent of data (Gbps of user traffic watching videos, gaming, etc.).

If you try to push all of that through the single eth0 provided by a conventional Pod, you run into a serious “noisy neighbors” problem. The massive User Plane traffic can saturate the interface and cause delicate Control Plane packets to be delayed or dropped (jitter and latency).

Additionally, in Telco environments we often need the Pod to communicate directly with specific external physical networks, use different VLANs for management, or support redundancy protocols that require layer-2 access.

With a standard Pod, we are trapped inside the cluster network, isolated from the physical world and mixing all traffic in a single tunnel. This is where the native Kubernetes design falls short and where we need something more powerful.

IV - Red Hat OpenShift Virtualization

Red Hat OpenShift Virtualization

Now that we understand the “single-door networking” problem, let’s talk about the platform where we will build our solution.

What exactly is OpenShift Virtualization?

In short, it is the capability that allows us to manage Virtual Machines (VMs) as if they were containers.

Forget about having one silo for traditional virtualization and another for containers. OpenShift Virtualization (based on the upstream KubeVirt project and using the KVM hypervisor) unifies both worlds. This means your VMs inherit all the superpowers of Kubernetes: scheduling, high availability, and — most importantly for our discussion — network management.

Under the Hood: How Does It Work?

When we install the OpenShift Virtualization Operator, several key components are deployed to manage the virtualization lifecycle (storage, compute, and networking):

1. CRDs (Custom Resource Definitions): The operator provides new native objects. The most important ones are VirtualMachine (the definition) and VirtualMachineInstance (the running VM). Thanks to this, we can create and manage VMs using oc or through the web console, just as we would with a Deployment.
2. The brain and the muscle: In the control plane lives the virt-controller, which orchestrates the entire lifecycle by watching the CRDs. On each worker node, a DaemonSet called virt-handler runs. Its job is to communicate with the virt-controller and execute the necessary actions on the physical node.

The “VM Pod” Concept

Here is where the magic happens. When you start a VM, OpenShift does not create a separate black box outside the cluster. Instead, it launches a Pod.

Inside that Pod runs a component called virt-launcher. Its job is to prepare the environment and start the libvirtd/qemu process that ultimately runs the virtual machine.

For OpenShift, it is just another Pod consuming resources. For us, it behaves like a full server running an operating system.

Preparing the Ground: Networking and NMState

As mentioned earlier, by default this “VM Pod” starts with a single network interface (eth0). For our Telco use case, we know that this is not enough. The final solution will be Multus, but Multus cannot perform magic if the underlying hardware is not ready.

Before we can connect a VM to a second or third network, the physical nodes (workers) must have those networks configured.

This is where Kubernetes NMState comes into play.

Red Hat implements NMState so we can configure node networking in a declarative way. There is no longer a need to SSH into every server and run Linux commands manually. Instead, we define an object called NodeNetworkConfigurationPolicy (NNCP) and the cluster applies the configuration automatically.

Take this example. Here we are preparing the node by creating a Linux bridge on top of an additional physical interface:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: br1-eth1-policy 
spec:
  nodeSelector: 
    node-role.kubernetes.io/worker: "" # Applies to all workers
  maxUnavailable: 3 
  desiredState:
    interfaces:
      - name: br1
        description: Linux bridge with eth1 as a port 
        type: linux-bridge
        state: up
        ipv4:
          dhcp: true
          enabled: true
          auto-dns: false
        bridge:
          options:
            stp:
              enabled: false
          port:
            - name: eth1 # Physical interface
    dns-resolver: 
      config:
        search:
        - example.com
        - example.org
        server:
        - 8.8.8.8        

In this snippet, we are telling the cluster: “Take the eth1 interface on your nodes and turn it into a bridge called br1.”

We could also configure bonds (LACP), VLANs, or SR-IOV depending on the performance requirements of our User Plane.

Once the policy is applied and our nodes have the interfaces ready, the foundation is in place. Now it’s time to use Multus to connect our virtual machines to these new networks.

At this point, our cluster is ready. We have applied the policy, the physical nodes have their interfaces prepared (bridges, VLANs, or SR-IOV configured), and the foundation is solid.

But we are missing the most crucial piece of the puzzle: our Virtual Machine is still trapped inside the cluster's default network. How do we physically hand over these new networks to the VM to isolate control traffic from user traffic at bare-metal speeds?

In Chapter 2 of this article, we will introduce Multus CNI to plug in these virtual cables, and we will assemble the Final Architecture to demonstrate how a production 5G Telco environment is deployed using pure Infrastructure as Code (IaC).

See you in the next part!

Wanna talk to one of our nerds? Go to https://www.netlabs.us/