What are Container Runtimes?

As more and more applications are being migrated to containers or built from the ground up with containers in mind, it’s essential to know what a container runtime is and how the various options fit together with the rest of your technology stack.

In order to start and successfully run on a machine, every container must have a properly installed and functional runtime. Container runtimes don’t care if the host is an EC2 instance, installed directly on a bare metal server, or even running on a Linux host in an LPAR on an IBM zSeries mainframe. The container runtime is a software package that knows how to leverage specific features on a supported operating system to create a space to run the specified container image.

How Does a Container Runtime Work?

Open Container Initiative (OCI)

The OCI is an organization backed by the Linux Foundation which manages the three primary specifications with which all modern container runtimes must comply:

• The actual container image specification
• How a container runtime can retrieve a container image
• How an image is unpacked, layered mounted and executed

These specifications are all based on Docker’s pioneering work. Docker donated them to the initiative because they saw the value in having an open and interoperable ecosystem.

Runtime Process

The basic steps of the container lifecycle as defined by the OCI runtime specification are as follows:

• The container runtime is asked to create a new container instance with a reference to where the container image can be retrieved as well as a unique identifier.
• The container runtime reads and validates the container image’s configuration.
• Next, the namespace and mount points are created, and any quotas are applied while unpacking and merging the container image layers. This way, the container’s file system is in place and ready to go.
• Then the start is issued, which launches a new process with its root filesystem set to the mount point that was created in the previous step so that the container can only see those specific files and can only operate within the specified quotas and security policies.
• At this point, a stop can be issued to shut down the running container instance.
• The final step that can be performed is to delete the container instance, which will remove all references to the container and clean up the file system.

Rarely does anyone directly interact with these low-level commands. Most people typically use an orchestration platform (like Kubernetes) or a high-level container runtime that provides a more user-friendly experience.

Types of Container Runtimes

Low-Level Container Runtimes

Any container engine that implements the OCI runtime specification is considered to be a low-level container runtime. These are the basic building blocks that make containers possible and do the actual unpacking, creating, starting, and stopping of the container instances:

• runC was created by Docker and donated to the OCI. It is the default container runtime that is used almost universally on Linux hosts.
• crun is an OCI-compliant runtime focused on being extremely small and efficient (with a binary size of about 300KB).
• runhcs is a fork of runC created by Microsoft and used on Windows to run Windows containers.
• containerd is on the line between low-level and high-level. It actually leans more toward a high-level container runtime because it provides an API layer on top of an OCI-compatible runtime, but we will include it here because you don’t really interact with it directly. There’s always an external layer that you use to interact with it, such as a CLI (like nerdctl or ctr) or CRI on Kubernetes.

High-Level Container Runtimes

• Docker is probably still the best-known container runtime platform in the mainstream. In reality, it is a nice set of tools that make developers’ lives easier. It’s wrapped around containerd, which is used to manage the actual container images and instances. Docker also has tooling that can interact with Kubernetes instead of directly with containerd if developers prefer to work that way.
• Podman was built by Red Hat to offer a more secure model than Docker’s original implementation. Podman (along with Buildah and Skopeo) aims to provide the same high-quality developer experience as the Docker toolset. Unlike Docker, Podman does not require a daemon and allows users to run containers without root privileges.
• CRI-O is a purpose-built runtime that is designed to adhere to the Kubernetes CRI (Container Runtime Interface) specification. It receives the CRI requests and can communicate with any OCI-compliant runtime (like runC).

Container Runtime Security

Most of the activities related to security happen outside the scope of the actual container runtime. Those activities include secure sourcing, scanning (SAST, IAST, etc.), fuzz testing the application, and applying policies (like network access control) to shape incoming and outgoing traffic.

The actual runtime leverages Linux kernel functionality (like cgroups and namespaces) to provide a level of isolation for the running container. While cgroups and namespaces provide protection in the form of usage governance, all containers do share the same kernel. This is not typically a large attack surface, but in the event that the right kind of exploit gets discovered and executed in a container, it will expose all running services on the host and may lead to further privilege escalations.

There are two approaches that can limit this: running the containers in a sandbox or running them in a virtualization framework.

• Sandboxed – Sandboxed container runtimes introduce a proxy layer between the running container and the actual kernel that intercepts and monitors the workload’s instructions to the kernel (like gVisor’s runsc). This is a great choice if you are running untrusted workloads (as Google does on its public cloud).
• Virtualized – Virtualized container runtimes launch the container image inside a stripped-down virtual machine. This is probably most commonly used if you run a Linux container on a Windows host, as that leverages Hyper-V. The other well-known example of this type of runtime is Kata Containers.

Container Runtimes and Kubernetes

Kubernetes is the most widely used container orchestration engine. It has an entire foundation and ecosystem that surround it with additional functionality to extend its capabilities through its defined APIs (one of which is the Container Runtime Interface (CRI), which defines what Kubernetes wants to do with a container and how). It utilizes the basic admin commands that you’d expect – like create, start, stop, update, and list. Higher-level container runtimes like Docker Desktop can do much more, including build, inspect, push, and sign container images.

Due to the combination of the CRI interface with the OCI specifications, it is possible to use essentially any compliant runtime in your Kubernetes cluster. Containerd is the most common runtime included with Kubernetes distributions, and the second most widely used would be CRI-O.

How Do I Choose My Container Runtime?

Choosing your container runtime will most often be driven by the suite of software that you are using to manage the entire environment. These management and orchestration solutions (like Kubernetes) will be opinionated, and they will either install the container runtime they want on their own or have it listed as a prerequisite.

The vast majority of solutions (not just Kubernetes) rely on containerd, including developer-friendly tooling like Docker Desktop. Exceptions include Red Hat OpenShift, which uses CRI-O, and Podman, which directly interacts with low-level container runtimes like runC. In addition, you may have isolation and security concerns that go beyond what containerd can offer, in which case you would want to consider other runtimes like gVisor or Kata Containers.