Preventing malicious use of Weave Scope

Intezer and Microsoft reported on Sept. 9 that TeamTNT hackers are deploying Weave Scope in compromised systems as an auxiliary tool in their intrusions. Weave Scope is a legitimate and powerful tool to manage server infrastructure that, once deployed, makes it easy to control all resources.

In this article, we will describe how this tool can be used maliciously, and how to add specific checks in your security set up to look for it.

Vulnerability Context

The vulnerability that TeamTNT used to originally gain entry was an unsecured Docker API endpoint exposed to the internet. After exploiting it, they then proceeded to deploy Weave Scope as their tool to scan the whole system.

The attack flow

  1. The attacker identified an exposed unprotected Docker API.
  2. They spun up a new Ubuntu container on the host running the exposed Docker daemon.
  3. The new container mounted the host filesystem, allowing access to sensitive information.
  4. They created a new privileged user they can use to SSH into the server.
  5. After connecting to the host, they downloaded and installed the Weave Scope application.
  6. They connected to the Weave Scope console over port 4040.
  7. This gave the attackers full visibility into the Kubernetes environment, allowing them to identify the sensitive assets and take over the environment.

Attack flow when exploiting Weave Scope

The impact of exposed Docker API and Weave Scope reach

An exposed Docker API allows any attacker to execute workloads in your infrastructure as root users. This way, an attacker can gain access to the host operating system, all of the running containers, and possibly all of the cluster’s infrastructure.

From there, Weave Scope is going to make it easier to scan all workloads to search for further vulnerabilities, exfiltrate any unsecured information, and execute additional workloads like cryptominers. They can also plant additional exploits, ready to be used in case Weave Scope is terminated and the Docker API endpoint gets secured.

Screenshot of Weave Scope topology graph
Screenshot of Weave Scope topology graph

Mitigating exposed Docker API port and Weave Scope usage

Exposed Docker API port

A full blown security audit of all of your infrastructure is the recommended action if you suspect you may be compromised, as no one knows what could have been installed with this exploit.

But for a quick check, first make sure the Docker API port is well protected and not accessible from a public IP. You can check it by reaching port 2376 (default Docker port) from outside with these commands:

export public_ip=<your_infra_public_ip>
nc -z $public_ip 2367 ; if [ "$?" = "1" ]; then echo "Port not exposed" ; else echo "Port exposed" ; fi

Any cloud managed container platform is going to use a secure by default approach. Make sure you follow their recommendations on how to connect to your workloads in a secure way.

If you have deployed Docker or Kubernetes by yourself, you must know that they follow the opposite paradigm. In them, everything is unsecured by default, and you have to make sure you add your own security to it. A secure Kubernetes deployment or Docker installation aren’t trivial tasks. Although Docker and Kubernetes automatically provide many management mechanisms out of the box, you still have to take care of their security and authentication configuration.

Falco, a CNCF incubating project, can help detect anomalous activities in cloud native environments. As an extra precaution, you can use a Falco rule like this one to detect any connection to the port 2376 from outside of the local network:

- rule: Connection to exposed Docker API on port 2376
  desc: Detected connection to exposed Docker API on port 2376
  condition: > 
    (inbound_outbound) and fd.sport=2376 
    and not fd.snet in (rfc_1918_addresses))
  output: >
    Detected connection to exposed Docker API on port 2376
    (command=%proc.cmdline connection=%fd.name user=%user.name
    container_id=%container.id image=%container.image.repository)
 priority: CRITICAL
 tags: [network]

Detect Weave Scope deployment

You should check if Weave Scope is already deployed in your Docker or Kubernetes infrastructure.

Checking all running images on Docker is easy with docker ps:

docker ps | grep weavescope

To get a nice list of all running images on Kubernetes, execute:

$ kubectl get pods -o jsonpath="{..image}" --all-namespaces |\
    tr -s '[[:space:]]' '\n' |\
    sort |\
    uniq -c

An image scanner can also help you detect the Weave Scope image, adding weaveworks/scope to a global block/disallow list.

You can also leverage Falco to make sure you prevent the image from running unnoticed on Docker or Kubernetes, with a rule like this:

- rule: Launch of Weave Scope network tool container
  desc: Detect launching Weave Scope network tool container
  condition: >
    container and
    container.image.repository startswith 'weaveworks/scope'
  output: >
    Detect launching Weave Scope network tool binary process running
    (command=%proc.cmdline connection=%fd.name user=%user.name
    user_loginuid=%user.loginuid container_id=%container.id
    image=%container.image.repository)
  priority: WARNING
  tags: [network]

Keep in mind, the image could be deployed with a different registry/image name combination than the original Weave Scope one. So you should add some checks that don’t depend on the image name.

For example, some image scanners allow you to detect specific files and folders. Remember, Weave Scope is a legitimate tool, so it’s not going to appear in a database of vulnerabilities. If you visit the Weave Scope image entry in Docker Hub, you can see that its entrypoint is:

ENTRYPOINT ["/home/weave/scope" "--mode=probe" "--no-app" "--probe.docker=true"]

So if we create an image scanning policy to check for that specific endpoint, we are going to detect this tool even if the name or source of the image is a different one.

The best way to do is to use a “like” comparison against “*/home/weave/scope*,” in case additional parameters change.

You will ideally set up this policy to be used with an admission controller in Kubernetes, preventing matching images from being deployed. You should also include an inline scan step in your CI pipelines, so non-compliant images are blocked from reaching your registry.

The last line of defense is to add a Falco rule to detect if the directory/binary combination is running in the kernel of the Docker host, or Kubernetes nodes, no matter the nature of the method used to deploy it:

- rule: Weave Scope network tool binary process
  desc: Detect the Weave Scope network tool binary process running
  condition: spawned_process and proc.cmdline contains 'weave/scope'
  output: >
    Detect launching Weave Scope network tool binary process running
    (command=%proc.cmdline connection=%fd.name user=%user.name
    user_loginuid=%user.loginuid container_id=%container.id
    image=%container.image.repository)
  priority: CRITICAL
  tags: [network, weave_scope]

Conclusions

An exposed Docker API entrypoint is a very dangerous threat vector, and combined with the powerful Weave Scope tool, it can deeply compromise your whole infrastructure very quickly.

Sysdig Secure can help you with its inline scanning, runtime security using Falco, admission controller, and image analyzer features. Try Sysdig today!

Stay up to date

Sign up to receive our newest.

Related Posts

Understanding and mitigating CVE-2020-8566: Ceph cluster admin credentials leaks in kube-controller-manager log

Understanding and mitigating CVE-2020-8563: vSphere credentials leak in the cloud-controller-manager log

Detecting CVE-2020-14386 with Falco and mitigating potential container escapes