Building Docker Compose Support on an event driven stack

In my previous post, I went over how our event-driven architecture allows us to rapidly ship new features. This post covers how we used this model to ship a new feature: Docker Compose support.

Docker Compose enables our customers to build environments on Runnable using the same configuration they use to deploy to production, staging, or wherever they currently use Docker Compose. The best part? They get this with no additional setup.

Approach

We broke the implementation of this feature down into two flows:

• Importing: Pulling in a team’s Compose file and using it to build Runnable environments.
• Updating: Applying changes to their environments when a user pushes an update to their Compose file.

Implementation

Runnable configurations consist of a Docker image, a run command, environment variables, and port mappings. At the highest level, we just needed to convert the Compose file to a Runnable configuration via an alternate flow of events and tasks. A ponos-based worker server handled the events and tasks, and a parser, which we lovingly named octobear, handled the actual conversion.

When a Compose file is imported from a repository, it triggers the following flow:

1. The `compose.requested` event is emitted to our event-driven system. Our worker server listens for this event and creates the following task:
2. `compose.parse-config` parses the Compose file and emits a `compose.parsed` event.
3. A `compose.cluster.create` task is created from the previous event. Here, we wrap back into our core flow; this task creates a cluster and emits the `cluster.created` event which is also emitted in our core flow.

When a Compose file is updated on a repository, it triggers the following flow:

1. We monitor `github.pushed` events to see if any were created from a Compose-configured repository. If so, we emit the following event:
2. `compose.cluster.pushed` checks to see if the Compose file was changed. If not, we simply emit the `cluster.updated` event which continues our core flow. If the Compose file was updated, we emit the following event:
3. `compose.cluster.config.updated` creates a task called `compose.cluster.sync` to sync our Runnable configuration state with the updated Compose file. Once complete, we loop back into our core flow by emitting `cluster.updated`.

Benefits

Our eventful infrastructure helped us ship Compose support faster and with little risk. We kept our core path untouched and were not tied down to our current application’s deployment cycle. The best part of this architecture is how easily we can expand on it to support other multi-container orchestration frameworks (like k8s). If you’re looking to create a new architecture plan, consider making it event-driven so you too can create apps at light speed with little risk.

Originally published at blog.runnable.com on January 17, 2017.