Forking Docker Not

It’s time, not to fork the Docker Project, but to focus on standardizing the container runtime with a clear scope set by the Open Containers Initiative (OCI) and collaborate on a common, vendor-neutral upstream.

To frame what I am writing here: I work at Red Hat and I am involved in our container efforts around the OpenShift Container Platform and Red Hat Enterprise Linux Atomic Host, however the below is my personal opinion and does not claim to represent a Red Hat consensus.

There has been growing tension in the broader Docker community over the last year or so, maybe longer. The conflict started to escalate earlier this summer, when Docker Inc used its controlling position to push Swarm, it’s own clone of Kubernetes-style container orchestration, into the core Docker project, putting the basic container runtime in a conflict with a notable part of its ecosystem. Docker Inc. then went on to essentially accuse Red Hat of forking Docker - at the Red Hat Summit no less. After that, Docker Inc’s Solomon Hykes came out strongly against the efforts to standardize the container runtime in OCI - an initiative his company co-founded.

Over the last week, it appears that those tensions have finally built to rupturing pressure: there are rumors of cloak and dagger meetings to discuss forks, an open debate about why the inclusion of Swarm in core Docker was a bad idea, and calls for forking the Docker project with the goal of creating a vendor-neutral upstream as part of the Cloud Native Computing Foundation (CNCF).

From my point of view the importance of this conflict can not be under-estimated. I see containers as the future of the Linux OS and application-centric IT. Because of the high stakes, this controversy puts the industry at risk of repeating mistakes of the past: let the Linux ecosystem fragment and compete in the low-level plumbing, rather than focusing on value to the end-users. And I see the root cause for this tensions in the aggressive way that Docker Inc is trying to control the Docker open source project, which was started under a permissive open source license when the company still was called DotCloud. Permissive licenses alone do not create a community and do not guarantee that a community’s best interests will be the priority when accepting pull requests or setting architectural direction. In the Docker community, Docker Inc. controls the architectural direction and uses it to aggressively change the direction of the Container Engine to the detriment of stability.

Don’t get me wrong. I have a lot of empathy for Docker Inc. and their technical leaders: I can absolutely relate to their struggle to maintain the integrity of their technical vision as well as to find a sustainable business-model that would justify the level of investment they received. Nothing I am writing here is meant to lessen the ingenuity Solomon Hykes demonstrated when combining aggregate packaging, layering and containers into a new model for application deployment, triggering what I personally consider a bigger shift of paradigm in computing than virtualization. And it does not lessen the commitment and excellence of the employees of Docker Inc. I have had the opportunity to interact with, and who, without exception, I highly respect. At the same time, this does not change the reality that having a single commercial party with its business interests control a widely adopted open source project is not viable. Or to quote Google’s Craig McLuckie: “.@docker must be allowed to innovate and profit (they have legitimately done amazing work!), but we need boring core infrastructure today”.

What has to happen in order to prevent fragmentation, is pretty clear, as there is excellent precedence: Linux was successful despite its commercial adoption to a large degree because Linus Torvalds remained an excellent technical leader who put the good of the project above all business interests. He established a culture of meritocracy that rewards good-faith technical contribution and nothing else. Unfortunately the Docker project has gone in the opposite direction.

An example for the perceived conflict is how some of Red Hat’s submissions are handled by Docker Inc. As Joe Fernandes from OpenShift recently pointed out, Red Hat has been one of the primary contributors to the Docker project from the early days and engages openly in the upstream community. Red Hat OpenShift Container Platform is a complete, fully open source container platform supported for production enterprise deployments and available today. It’s based on Red Hat Enterprise Linux, the Docker project, OCI standards and Kubernetes. Yet on a regular basis, Red Hat patches that enable valid requirements from Red Hat customer use cases get shut down as it seems for the simple reason that they don’t fit into Docker Inc’s business strategy.

These patch submissions should be uncontroversial in an open source project: enabling multi-service pets and legacy applications by allowing systemd to run inside a container; enabling the import of subscription secrets from the host; allowing the sysadmin on the local system to control the search path and registries for containers; having a federated namespace that is not Docker Inc-centric; enabling features such as redirecting container-layers to the authoritative source on all operating systems; etc. The last one is especially interesting as it’s a feature that Docker has exclusively enabled for Microsoft while at the same time refusing Red Hat and other Linux vendors to use it - although the use case is very much the same.

Most vendors, end-user companies, and community members buy into the value of a common standard for containers. So you can imagine that there is quite some grumbling, and a strong interest in standardizing the key aspects of the container runtime in OCI to achieve some vendor-neutrality. That said, I am not aware of any concrete plans to fork the Docker project, and none of the patches Red Hat or the other downstream distributors and integrators ship constitute a fork. Like others, Red Hat has remained committed to contributing to the Docker project with an upstream-first approach, carrying patches only after trying to get them integrated into the mainstream repo. Carrying patches against open source projects is what downstream integrators and distributors do. It’s the whole point of open source licenses to give you the right to create modified versions of code and distribute them. Dan Walsh has published a blog explaining the add-on patches Red Hat carries - and their upstream submission status - in detail.

I do think that it’s time for the different parties who carry patches to merge their add-ons to the Docker project into a single, joint repository with some simple governance to reduce the redundancy and improve collaboration. Perhaps that would also help to get a better read on what people outside of Docker Inc care about. I’d suggest to move this vendor-add-on repository under the OCI umbrella. At the same time, this should not fork the Docker project and instead continue to follow an upstream-first approach trying to get the patches accepted into mainstream Docker. And it’s only a band-aid.

The longer-term solution is a proper scope for OCI covering the full end-to-end use case of the core container runtime as Brandon Philips from CoreOS has proposed. And while I personally consider the orchestration layer the key to the container paradigm, the right approach here is to keep the orchestration separate from the core container runtime standardization. This avoids conflicts between different layers of the container runtime: we can agree on the common container package format, transport, and execution model without limiting choice between e.g. Kubernetes, Mesos, Swarm.

Key to the success of OCI will be to have a common open upstream - this is open source and code beats whatever standard people define. Primarily because of technical reasons, my recommendation is to - instead of forking the existing Docker project - continue on the path that Docker Inc themselves have started with RunC: out of OCI provide a reference implementations as a set of distinct modules covering the core-runtime components. Individual projects and vendors can then choose to use these modules or implement their own compliant alternatives may it be OCID, Docker, rkt or others. Dan Walsh presented an example of this runtime at Flock 2016. The governance for this should be lightweight, focused on meritocracy based on technical contribution and not on company control or pay to play. Follow the Linux example!

My hope is that Docker Inc will react to the controversy by adopting a win-win mentality, rather than continuing to control the project and push their business-agenda against all other contributors. The debate shows that unless we find a way for the standard container runtime to evolve in a model that is inclusive of the requirements and submissions from a large number of contributors, we will see fragmentation. So we are at a pivot point on whether we will have a common standard for the container runtime or not.

DON’T PANIC!

PS: The improbable whale infographic was taken from a t-shirt design by maped. Go order some t-shirts.