The Container Ecosystem Project

By Knox Anderson - OCTOBER 21, 2015


The ecosystem of awesome new technologies emerging around containers and microservices can be a little overwhelming, to say the least. We thought we might be able to help: welcome to the Container Ecosystem Project. The goals of this project are (1) to clearly lay out the different types technologies that make up the growing container ecosystem and the microservices technology stack – starting from the lowest levels of core container technology, and rising up through layers of abstraction to full-blown container platforms and support tools – and (2) to put forth the latest and greatest examples of each type of technology. This project is a living document – please suggest edits to the github repo and see below for more info.

Table of contents

The Container Ecosystem (View on Github)

Core Container Technologies

Use these tools to run a small number of containers on a single host
Docker open source CoreOS open source Other open source Commercial
Container specifications
An abstract definition of a standard “container”, allowing an ecosystem of technologies to support a standard container with potentially multiple, interchangeable runtime implementations
  • AppC (deprecated): CoreOS is now supporting the OCI
Container runtimes
This is your actual running container (essentially an abstraction of Linux kernel components like namespaces and cgroups that allow virtualization on top of a shared kernel)
  • runc: Docker’s container runtime, now donated to the OCI as the initial implementation of the standard; essentially a repackaging of libcontainer
  • libcontainer: a Linux container library; enables and abstracts interactions with Linux kernel components to create and control containers
  • rkt: CoreOS’s container runtime; initially an implementation of the AppC specification, which is now being rolled into the OCI spec
  • LXC: a Linux container library; originally utilized by runc until release of libcontainer
  • OpenVZ: a Linux container library
Container management
These tools abstract low level control of your container runtime adding further functionality and usability
  • Docker Engine (aka “Docker”): the core of Docker and its primary interface; creates and runs Docker containers; includes:
    • Docker daemon: runs as a process on the host machine and provides an API that abstracts basic container control functions
    • Docker client: a CLI for interacting with the Docker daemon
  • rkt CLI: rkt’s container management functionality is delivered on-demand by a binary, rather than a daemon background process
  • LXD: daemon and UI for LXC
  • libvirt: container and virtualization mgmt library that supports LXC, OpenVZ, and a variety of hypervisor technologies
Container definition
These tools allow you to define specific containers, so they can be saved, shared and reproduced
  • Docker image: a template representing a fully configured container; Docker container runtimes are created from these images; images are created with Dockerfiles and shared over registries
  • Dockerfile: text file containing all the commands needed to build a Docker image
Repositories for storing and sharing container images
  • Docker Registry: open source Docker image registry that can be hosted in your own environment
  • Hosted
  • On-premise
Operating systems
OS’s that are designed for hosting containers
  • boot2docker (basically deprecated by Docker Machine): minimalist Linux for running Docker on PC and Mac in a VM; now used by Docker Machine in certain environments
  • CoreOS: minimalist OS built for running distributed, containerized apps; includes etcd and fleet
VM management
These tools help you manage the host virtual environments in which you run your containers
  • Docker Machine: creates and manages host VMs running Docker, including local VMs (eg. VirtualBox) and cloud VMs (eg. Amazon AWS, Google GCP)
  • Hashicorp Vagrant: creates pre-configured VMs for dev environments based on a variety of “Providers” (virtualization technologies) including Docker containers
  • Hashicorp Otto: extends Vagrant to deploy and manage VMs across many platforms

Distributed Container Technologies

Use these technologies to run applications on a distributed cluster of containers
Docker open source CoreOS open source Other open source Commercial
These tools manage placement of new containers across abstracted underlying resources
  • Docker Swarm: designed to extend Docker API to a cluster; includes scheduling and service discovery
  • fleet: low level orchestration included in CoreOS; supports basic scheduling; can be used to bootstrap Kubernetes for higher level orchestration
  • Chronos: framework for scheduling on Mesos
Cluster definition
These tools allow you to define and manage a cluster of dependent containers as a single composable entity
  • Docker Compose: text files used to define and configure a distributed application across a cluster of Docker containers
  • fleet unit file: fleet uses a specialized version of systemd unit files to define a distributed application across containers
Service discovery / Distributed configuration storage
These tools allow applications within different containers to discover each other and share configuration information (eg. IP addresses or application settings); usually implemented as a globally distributed key-value store
  • Docker Swarm comes with built in service discovery, but can also use etcd, Consul, Zookeeper
  • etcd: globally distributed key-value store; included with CoreOS for service discovery
  • Marathon: framework for initializing long running jobs on Mesos; includes service discovery and cluster management functionality
  • Hashicorp Consul: service discovery, key/value store, and cluster health checking; uses Serf
  • Apache ZooKeeper: globally distributed key-value store
Dynamic configuration management
These tools let you dynamically update application settings based on changes to your distributed key-value store in applications that don’t natively support this
  • confd: originally built for etcd, but now supports Consul and ZooKeeper

Container Platform Technologies

Use these technologies as complete platforms for running distributed applications across clusters of containers
Docker open source CoreOS open source Other open source Commercial
Container orchestration platforms
These platforms include or abstract away all of the core functionality (listed above) needed for container cluster management (“orchestration”), including container management, scheduling, cluster definition, and service discovery
  • Docker Swarm, Compose, and Machine can all run together to create a complete orchestration platform (still beta); Docker Swarm can also support more advanced orchestration tools like Kubernetes
  • Apache Mesos: mature, highly scalable service that abstracts a pool of underlying resources and distributes “tasks” (including Docker images) from various application frameworks; uses Marathon and Chronos to add cluster management, scheduling, and service discovery; also can support Kubernetes
  • Kubernetes: orchestration platform designed specifically for running microservices on clusters of containers; includes scheduling, cluster management and service discovery through abstractions such as “pods”, “replication controllers (RCs)”, and “services”; originally from Google, now donated to the CNCF
  • Hashicorp Nomad: uses Consul
Hosted container platforms
These platforms offer container hosting and orchestration as a service
Container platform management
These technologies add further abstracted management and control layers to distributed container environments, often through GUIs
  • Project Orca: opinionated management GUI built on top of full stack of Docker technologies; still alpha
Container-based PaaS
These platforms further abstract container-based infrastructures by managing application code deployment and offering PaaS-like user experiences

Container-Native Support Technologies

Use these additional container-native tools to support your container-based infrastructure
Docker open source CoreOS open source Other open source Commercial
  • Docker port expose: Docker feature that links a container port to a host port
  • Docker linking: Docker feature offering a basic connection between containers on the same host
  • libnetwork: advanced container networking library (still “under heavy development”)
  • flannel: overlay network built using etcd that gives each host a separate subnet for its containers
  • Weave: overlay network that puts all containers in a distributed system onto a single virtual network; also includes service discovery functionality
  • Calico: layer 3 virtual network that provides each container with an IP address
Monitoring / Visibility
  • sysdig: CLI for deep system/containers visibility; includes curses-based “csysdig” interface
  • cAdvisor: basic container metrics exporter from Google; includes web GUI; Heapster adds Kubernetes support
  • Weave Scope: container network topologies
  • Sysdig Cloud: uses sysdig; includes web-based UI, application topologies, and support for all major container formats and orchestration platforms
Data layer
  • CusterHQ Flocker: data volume manager for running stateful services like databases in containers
Log management
  • logspout: log router for Docker containers
Getting started aides

About the Container Ecosystem Project

Here at Sysdig, the container-native visibility company, we talk to a lot of people in the container ecosystem: both consumers and producers of technology. And wow, there is a LOT of cool technology out there – and so much more coming out all the time. It can be hard to keep up with, even if you’re a seasoned expert, much less as a curious newcomer just trying to figure out where to start. There are plenty of great guides out there for various container technologies and use cases (see below for some links). But we had yet to find a clearly organized survey of the different core technologies that make up the container ecosystem and the typical microservices stack. So we decided to make one: the Container Ecosystem Project.

The goal of this project is to clearly lay out the different core technologies that might be important for anyone interested in containers and microservices – starting from the lowest levels, and rising up through layers of abstraction to full-blown container platforms. For each type of technology (broken into rows), we’ve tried to provide a brief description (see the left column), as well as list examples currently available for that technology (see the other columns). We’ve separated out open source solutions from commercial offerings, and two of the leading open source container technology producers, Docker and CoreOS, each got their own column. Throughout the doc, we’ve tried to mark beta technologies and parent technologies accordingly. Ideally, this document can introduce you to the microservices stack, and give you some keywords that you can then go research further on your own to learn more – but at least you’ll hopefully have the big picture from here.

This framework is not, of course, a perfect science, but we’ve done our best to create MECE categories by row, and to put each technology in the most appropriate row. We are almost certainly missing many great technologies, and many technologies listed here do not yet have perfect descriptions. This will be a work in progress. If you have any suggested edits, please tweet us or submit a pull request. We’ll do our best to keep this document up to date and prune off deprecated or abandoned technologies as the ecosystem evolves.

That’s all for now. I hope this can be a useful resource for the community!

Update: you can also comment on Hacker News here.

Further Reading

Subscribe and get the latest updates