Why and how to secure pods with Kubernetes security contexts

How can you ensure that each resource in Kubernetes has the permissions it requires, while at the same time avoiding over-permissioning? In other words, how can you define permissions on a granular basis that adheres to the principle of least privilege?

The answer is Kubernetes security context. Security context is a tool that allows admins to define security-related parameters on a resource-by-resource basis. As such, it makes it possible to assign each resource the specific permissions that it needs to access resources on the host server while denying access to those that it doesn’t specifically require.

This article provides an overview of Kubernetes security context, including how it works, how to define a security context, and which limitations exist when working with security context in Kubernetes.

What is Kubernetes security context?

In Kubernetes, a security context defines privileges for individual pods or containers. You can use security context to grant containers or pods permissions such as the right to access an external file or run in privileged mode.

Internal vs. external security contexts

Kubernetes security context is a bit complicated in the sense that some of the rules that you can define are enforced internally via Kubernetes itself, whereas others integrate with external security context tools, namely, AppArmor and SELinux.

You can think of Kubernetes security context as a way to define certain permissions for pods and containers, as well as to integrate Kubernetes with external security tools that run on the host rather than in Kubernetes itself.

Security contexts vs. role-based access

Security context is similar to, but distinct from, Kubernetes role-based access control (RBAC). RBAC controls API access and its resources, while security contexts handles runtime privileges inside containers.

Other key differences include:

• Resource scope: RBAC can be applied to a variety of Kubernetes resources, such as pods, Kubernetes nodes, and even entire clusters.
• Actions: RBAC can grant a variety of permissions based on verbs that admins can define within RBAC policies. Security context is more restrictive in that it only allows admins to assign specific types of predefined capabilities, like running in privileged mode (although security contexts are more flexible if you define rules using SELinux or AppArmor).
• Extensibility: As noted above, security contexts can be extended via integrations with external frameworks, including SELinux and AppArmor. Kubernetes RBAC can't use external tools to define policies.

So, think of security context as a way to define additional types of security permissions for containers and pods that RBAC can't manage. For most Kubernetes environments, you'll want to use security context and RBAC at the same time because they complement each other.

How to use Kubernetes security contexts

Using security contexts in Kubernetes is straightforward (especially if you're working only with internal security context and not integrating with SELinux or AppArmor). You simply include a block of security context code within the deployment file that you create when deploying a pod.

For example, the following block tells Kubernetes to run a pod with a user ID of 1000. It also assigns a group ID of 2000 to all containers in the pod:

spec:
 securityContext:
   runAsUser: 1000
   runAsGroup: 2000
   fsGroup: 2000
   runAsNonRoot: true

Unlike RBAC, security context doesn't require you to define multiple types of files (like Roles and RoleBindings) in order to enforce a security rule. You just need to add the requisite security context code when you declare a deployment, and Kubernetes will automatically enforce the rules from there.

For a full overview of types of permissions you can assign (or deny) with security context, refer to the Kubernetes documentation. In addition to the permissions related to user and group IDs, most admins will find value in the parameters that make it possible to let the pod or container run in privileged mode.

A container running in privileged mode has almost all of the same access rights to kernel-level resources on the host as a process that runs as root, so you'll typically want to disallow privileged mode. Instead, define permissions on a granular basis by, for example, allowing the container or pod to bind to a certain port or execute certain external binaries, while otherwise denying access to resources outside of the container.

Working with external security contexts

Enforcing security context based on external tools like SELinux and AppArmor requires a bit more work. Here are the basic steps.

Load the SELinux or AppArmor module

First, you need to make sure that the kernel module associated with the framework you are using (i.e., SELinux or AppArmor) is installed and loaded on the node or nodes that will host your containers or pods.

Since in most cases Kubernetes automatically assigns pods or containers to nodes, you won't know ahead of time which node will host which container or pod. As a result, you'll typically need to install AppArmor or SELinux on every node in your cluster if you want to define security contexts via one of these frameworks.

Load a profile

After loading the modules, you need to load the AppArmor or SELinux profile that you'll use to define permissions. For example, this AppArmor profile denies the ability to write files:

Save the profile in the node's filesystem at a location that Kubernetes can read.

Again, since you probably don't know which node will run which containers or pods, you'll need to load the profile on each node in your cluster. For efficient ways to do this (i.e., in order to avoid having to SSH into each node and apply the profiles manually), check out the Kubernetes documentation on node profiles.

Apply the profile in Kubernetes

Finally, add a security context block to your deployment file that tells Kubernetes to reference the external security context profile and apply it to the deployment:

apiVersion: v1
kind: Pod
metadata:
 name: apparmor-demo
spec:
 containers:
 - name: hello
   image: nginx
   securityContext:
     appArmorProfile:
       type: Localhost
       localhostProfile: path/to/policy-file

Apply the declaration with:

kubectl apply -f ./deployment.yml

Now, Kubernetes will enforce the policy you configured through AppArmor or SELinux just as it would a security context that you configure directly inside a deployment file.

Limitations of security contexts

Kubernetes security context is a powerful tool for defining certain types of permissions on a granular basis. However, it is currently subject to some significant limitations.

Limited Windows support

Many security context fields remain Linux-specific and Windows support remains limited and differs in implementation. Kubernetes does support Windows containers but has different security controls.

Limited to pod or container-level security

Security context can only define permissions for pods or containers. It can't control privileges at other layers of your stack.

Of course, most of the rules that you can apply with security contexts would only make sense when applied to containers or pods. It wouldn't make sense to tell a node to run in unprivileged mode, for instance.

Still, the point here is that security context is a tool only for addressing security issues at the pod or container level. You'll need other tools (like RBAC) to secure nodes, users, service accounts, and the like.

Inefficient tooling

As we've noted, a major limitation of some security contexts (specifically, those that use SELinux or AppArmor profiles) is that they require the deployment of external resources on every node in your cluster. While there are ways to automate this process, merely setting up the automation takes a fair amount of effort. Deployment could also be messy if you have nodes running different Linux distributions, in which case you may need to customize your AppArmor or SELinux setup for each distribution.

One can hope that tools to simplify policy profile deployment across nodes will appear in the future. But for now, don't underestimate how much effort it will take to set up the nodes.

Despite these shortcomings, security contexts are an important resource for plugging potential access control gaps in Kubernetes clusters. Although they are limited in scope and are not a complete access control solution unto themselves, you should take advantage of security contexts as a way to bolster the overall security of your cluster.

Secure your Kubernetes deployments with Sysdig

Sysdig container and Kubernetes security is a solution designed to move at the speed of cloud to keep up with different cloud threats and vulnerabilities that attackers exploit. Get real-time deep visibility, understand what risks to prioritize, and investigate threats quickly.