How to solve networking problem for nested containers

Integration of OVN with Kubernates ( In Progress )

OVN(Open Virtual Network) provides network visualization to containers. OVN's integration with Kubernetes works in two modes - the "underlay" mode or the "overlay" mode.

In the "underlay" mode

OVN requires a OpenStack setup to provide container networking. In this mode, one can create logical networks and can have k8 pods running inside VMs,  independent VMs and physical machines running some stateful services connected to the same logical network.

(For this mode to work completely, we need distributed load-balancer suppport in OVN, which is yet to be implemented, but is in the roadmap.)

This mode requires that you have a OpenStack setup pre-installed with OVN providing the underlay networking.

It is out of scope of this documentation to describe how to create a OpenStack setup with OVN.

http://docs.openstack.org/developer/networking-ovn/

In the "overlay" mode

OVN can create virtual networks amongst k8 pods running on multiple hosts. In this mode, you do not need a pre-created OpenStack setup. (This mode needs NAT support. Open vSwitch as of version 2.4 does not support NAT. It is likely that Open vSwitch 2.5 or 2.6 will support NAT)

For both the modes to work, a user has to install Open vSwitch in each VM/host that he plans to run his containers.

Kuryr Project ( In Progress )

Project Kuryr makes networking available to containers through Neutron. This all brings together how Neutron networking benefits containers like it does virtual machines. Kuryr is a Docker network plugin that uses Neutron to provide networking services to Docker containers. It provides containerized images for the common Neutron plugins.

Kuryr plans to make use of vlan-aware-vms, however this is not yet complete upstream for now. In the future, it would provide a shared API abstraction to allow modeling containers that reside inside OpenStack VMs.

Thereby, the underlying network provider can implement your desired virtual network topology without a 2nd layer of overlays.

GitHub - openstack/kuryr: Docker remote driver for OpenStack Neutron

What's about Magnum?

Magnum (containers-as-a-service for OpenStack) provisions containers inside Nova instances and those instances use standard Neutron networking.

These containers are referred to as nested containers.

Currently, there is no integration between Magnum resources and Neutron and the nested containers are served networking outside of that provided by OpenStack (Neutron) today.

OpenShift and Neutron Integration

We have no plant to integrate with neutron directly today. And there is no concrete plans yet.

Only option today is to use flannel with host-gw enabled (ip routes on host machine) which is experimental.

OpenShift Commons Briefing #29: Integrating OpenShift on OpenStack - YouTube