A recent CNCF-sponsored Kubernetes security audit uncovered CVE-2019-11246, a high-severity vulnerability affecting the command-line kubectl tool. If exploited, it could lead to a directory traversal, allowing a malicious container to replace or create files on a user’s workstation. This vulnerability stemmed from an incomplete fix of a previously disclosed vulnerability (CVE-2019-1002101).
CVE Overview
The CVE consists of two phases:- Crafting a malicious tar binary seeded in a container.
- The execution of vulnerable
kubectl cpcommand
Are you vulnerable?
Runkubectl version --client and if it does not say client version 1.12.9, 1.13.6, or 1.14.2 or newer, you are running a vulnerable version.
How do you upgrade?
Follow installation instructions here https://kubernetes.io/docs/tasks/tools/install-kubectl/
Not every installation guide will provide up-to-date kubectl versions at the time of this announcement. So, always confirm with kubectl version.
CVE Exploit Detection
In this blog post, we’re going to show you how to use Falco, a CNCF open source project, to detect both phases of the exploit described in this CVE. Falco is an open source container security monitor designed to detect anomalous activity in containers and hosts. Falco, originally a Sysdig project, taps into system calls to generate an event stream of all system activity. Falco’s rules engine then allows you to create rules based on this event stream, allowing you to alert on system call events that seem abnormal. Falco’s rich language allows you to write rules at the host level and identify suspicious activity.1. Use Falco to detect the replacement of /bin/tar with a malicious binary
- rule: Write below monitored dir
desc: an attempt to write to any file below a set of binary directories
condition: >
evt.dir = < and open_write and monitored_dir
and not package_mgmt_procs
and not coreos_write_ssh_dir
and not exe_running_docker_save
and not python_running_get_pip
and not python_running_ms_oms
and not google_accounts_daemon_writing_ssh
and not cloud_init_writing_ssh
and not user_known_write_monitored_dir_conditions
output: >
File below a monitored directory opened for writing (user=%user.name
command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline gparent=%proc.aname[2] container_id=%container.id image=%container.image.repository)
priority: ERROR
tags: [filesystem, mitre_persistence]
Output:
10:16:16.067583414: Error File below a known binary directory opened for writing (user=root command=cp /tmp/tar.hacked /bin/tar file=/bin/tar parent=bash pcmdline=bash gparent=containerd-shim container_id=404b298fb6ee image=nginx) k8s.ns=default k8s.pod=nginx container=404b298fb6ee k8s.ns=default k8s.pod=nginx container=404b298fb6eeFrom the falco rule alert output, there is an attempt to overwrite the
/bin/tar file with a file named tar.hacked from /tmp folder. This seems really suspicious. The output also gives rich detail about the container, image, pod, namespace information etc.
2. Use Falco to detect vulnerable kubectl cp:
- macro: safe_kubectl_version
condition: (jevt.value[/userAgent] startswith "kubectl/v1.15" or
jevt.value[/userAgent] startswith "kubectl/v1.14.3" or
jevt.value[/userAgent] startswith "kubectl/v1.14.2" or
jevt.value[/userAgent] startswith "kubectl/v1.13.7" or
jevt.value[/userAgent] startswith "kubectl/v1.13.6" or
jevt.value[/userAgent] startswith "kubectl/v1.12.9")
# CVE-2019-1002101
# Run kubectl version --client and if it does not say client version 1.12.9,
1.13.6, or 1.14.2 or newer, you are running a vulnerable version.
- rule: K8s Vulnerable Kubectl Copy
desc: Detect any attempt vulnerable kubectl copy in pod
condition: kevt_started and pod_subresource and kcreate and
ka.target.subresource = "exec" and ka.uri.param[command] = "tar" and
not safe_kubectl_version
output: Vulnerable kubectl copy detected (user=%ka.user.name pod=%ka.target.name ns=%ka.target.namespace action=%ka.target.subresource command=%ka.uri.param[command] userAgent=%jevt.value[/userAgent])
priority: WARNING
source: k8s_audit
tags: [k8s]
Output:
09:51:26.488577024: Warning Vulnerable kubectl copy detected (user=kubernetes-admin pod=nginx ns=default action=exec command=tar userAgent=kubectl/v1.13.4 (linux/amd64) kubernetes/c27b913)As the CVE fix describe, kubectl version 1.13.4 suffers the vulnerability which should catch DevOps’ attention so that they can setup the patch plan accordingly. This highlights the potential of the Falco engine to accurately identify malicious activity occurring in containers and Kubernetes environments.
Things to consider
As some attacks come in multiple phases e.g. bitcoin mining, it will be very important to correlate malicious activities to provide a better insight into the attack. In thekubectl cp vulnerability exploit example above, both /bin/tar file replacement and kubectl cp happened inside the same pod. Adding the two events up based on pod name makes perfect sense to reveal a true kubectl cp vulnerability exploit.