THREAT ALERT: Crypto miner attack – Sysrv-Hello Botnet targeting WordPress pods

The Sysdig Security Research team has identified a Cryptominer attack hitting a Kubernetes pod running WordPress, related to the recent Botnet Sysrv-Hello. The goals of the attack were to control the pod, mine cryptocurrency, and replicate itself from the compromised system.

In particular, the attackers targeted a misconfigured WordPress to perform initial access. They later tried to stop other potential malware, installed and executed a cryptominer, and finally, tried to replicate itself (in our honeypot infrastructure and in the internet).

What is noteworthy about this attack is that the binary hash of the attack script and the cryptominer were very recent, and the registries on malware databases show that very few people have detected it. So, most security software wouldn’t have been able to detect the attack, and you would have to rely on detecting its behavior to get alerted.

This attack confirms what we’ve been seeing for quite some time: Botnet and Crypto miner attacks are on the rise, and they come in many different forms. The fact that crypto currency prices crashed early in the year is not making things better. After all, your infrastructure is still free money for the attackers.

In this article, we will cover the characteristics of the specific attack recorder, the details of this particular attack, and how to protect your infrastructure against it.

What is Sysrv-Hello Botnet?

The Sysrv-hello botnet is a Windows and Linux infection first identified in late December 2020, that exploits multiple vulnerabilities and is deployed via shell scripts.

Sysrv-hello’s operators are constantly updating and changing the botnet to keep up with their needs. Since the first identification, attackers performed several changes in the shell scripts that install Sysrv-hello implant, which is how the executable malware gets deployed on host systems.

In their latest changes, the botnet has new features and exploits for replication. The most important new feature is the ability to download a Monero-like miner and start mining cryptocurrency.

Anatomy of the attack

Let’s start with a quick overview of the attack and the main steps executed, from the crypto mining activities to the replication from the compromised pod.

The attack uncovered went as follows:

1. The entry point was a WordPress set up with the admin login configured to default credentials. This installation resides in our honeypot account. Once breached, the attacker was able to download and execute the script on the machine.
2. The script tried to:
   1. Turn off the firewall and perform other network changes to have full internet access.
   2. Modify troubleshooting binaries to avoid detection (top binary).
   3. Download all the needed components to run the malware (curl, crontab, etc.).
   4. Remove competing mining programs by monitoring the ports commonly used by other mining families.
   5. Download and execute uninstall scripts to get rid of monitoring tools in the host.
3. With all the ground prepared, the executed script downloaded the sys.x84_64 binary from the attacker machine and executed it. The main goals of this binary are: mining cryptocurrency and exploiting different vulnerabilities on various public available services to replicate itself.
4. The script proceeds, gathering information and secrets from the compromised pod.
5. Once all the information is gathered, the attackers use it to attack other internal machines, and replicate itself internally over the cluster network.

Keep reading to discover how this process works in detail, the exact commands that are run, and the communication between the servers.

At the end, we’ll cover how to detect this attack with open source Falco.

How did we uncover the attack, and link it to Sysrv-Hello botnet?

#1 A hacked WordPress and a script executed

The attacker gained initial access to our honeypot WordPress using the default credentials, and they hacked the website successfully.

After breaking in, the first action of the attackers was to download and execute the following script from the internet:

Analyzing the script, we can see it does different activities, starting with usual operations such as checking internet connection and adding domains related to cryptomining to /etc/hosts.

A more curious step is the top binary substitution with a custom script. The goal is to remove the spawned cryptominer processes from the command output to avoid detection.

The script is also worried about persistence and it’s checking if the crontab binary is available in the system. In this case, since crontab wasn’t available, the script installed it:

The real persistence is performed in a second step by a binary we are going to talk about shortly.

Before actually starting the “real” malware binary, the script also tried to uninstall monitoring tools with the goal of avoiding detection. In the screen below, we can see the script downloads and executes uninstall scripts, trying to remove tools like alyun.service and bmc-agent.

#2 Cryptominer executable downloaded and executed

The script proceeds by downloading and executing the cryptominer binary by using nohup.

Checking the binary downloaded, we can see that it is actually a recent version with just a few security tools able to detect it.

The binary downloaded is written in Golang and is available for Linux and Windows with the following names:

• sys.x86_64 – Linux version
• Sys.exe – Windows version

Let’s deep dive into the binary and the two main processes spawned to understand their activities. For this article, we are going to use the linux version.

The Sys.x86_64 binary

Once the script executed the binary, it started performing various actions on the pod with three main goals:

• Obtaining persistence on the container.
• Running a cryptominer.
• Replicating itself through exploiting other public services over the internet.

Before starting its duties, the binary opens a random port on the host or pod as a mutex port. This port is used by the malware to check if there is another instance of itself already running on the host or pod. In this way, in case the port is already open the malware won’t perform any action.

The binary tried to connect to the port 53055 to see if the same malware is already up on the pod.

connect res=-115(EINPROGRESS) tuple=127.0.0.1:36988->0.0.0.0:53055

Since the connection failed, the binary opened the port and started with the activities.

listen fd=3(<4t>:::53055) backlog=128

It proceeds with persistence, creating a cronjob on the pod to get persistent in the new environment.

* * * * * /home/ec2-user/.ssh/0k8gxs

The binary then spawned two main processes inside our pod:

• Kthreaddk process – the miner process.
• Random name process – send exploits over the internet to replicate itself.

Syscall Inspection of the kthreaddk process (crypto miner)

The kthreaddk process is mainly focused on the mining activities. Analyzing the sycalls generated, we can see the config file loaded to start the mining activities.

data={.    "api": {.        "id": null,.        "worker-id": null.    },.    "autosave": false,.    "use_nicehash": true,.    "background": true,.    "randomx": {.        "init": -1,.        "mode": "auto",.        "1gb-pages": false,.        "rdmsr": true,.        "wrmsr": true,.        "cache_qos": false,.        "numa": true,.        "scratchpad_prefetch_mode": 1.    },.    "cpu": {.        "enabled": true,.        "huge-pages": true,.        "huge-pages-jit": false,.        "hw-aes": null,.        "priority": null,.        "memory-pool": false,.        "yield": true,.        "max-threads-hint": 100,.        "asm": true,.        "argon2-impl": null,.        "astrobwt-max-size": 550,.        "cn/0": false,.        "cn-lite/0": false,.        "kawpow": false.    },.    "donate-level": 0,.    "donate-over-proxy": 0,.    "log-file": null,.    "pools": [ { "url": "194.145.227.21:5443" } ],.    "retries": 5,.    "retry-pause": 5,.    "syslog": false,.    "user-agent": null,.    "verbose": 0,.    "watch": false,."pause-on-battery": false.} 

The process started communicating with the pool to get the jobs to work with. In this particular case, the pod opened a connection with the same IP we saw before, 194.145.227.21, using the port 5443. It looks like the connection with the pool is proxied by the attacker machine (C2) to avoid detection based on domains or ports.

The payload to log into the miner server pool, as well as negotiating mining mechanism, was:

data={"id":1,"jsonrpc":"2.0","method":"login","params":{"login":"x","pass":"x","agent":"XMRig/6.4.0 (Linux x86_64) libuv/1.38.1 gcc/9.3.0","algo":["cn/1","cn/2","cn/r","cn/fast","cn/half","cn/xao","cn/rto","cn/rwz","cn/zls","cn/double","cn-lite/1","cn-heavy/0","cn-heavy/tube","cn-heavy/xhv","cn-pico","cn-pico/tlo","cn/ccx","rx/0","rx/wow","rx/arq","rx/sfx","rx/keva","argon2/chukwa","argon2/chukwav2","argon2/wrkz","astrobwt"]}}

The successful response from the pool with the info negotiated was:

data={"jsonrpc":"2.0","id":1,"error":null,"result":{"id":"5ccc9a1e092bd09e","job":{"blob":"0e0eeda6ce8806d9152b51d43091e17f30a12f09814b5d732d1d6a077db35a6a49ca1e619ee56700000092b0c254968b591363e6bad84a31d4360ac166929841dc424b538cea75b291e1c233","job_id":"9s1","target":"b2df0000","algo":"rx/0","height":2424652,"seed_hash":"ecd3a12c863b63d19b9558d0554ce42e9b5eeceba9c08c7da75f409f1f2980a1"},"extensions":["algo","nicehash","connect","tls","keepalive"],"status":"OK"}}. 

The miner is also executing the keepalive action to be sure it is still up and running.

data={"id":3,"jsonrpc":"2.0","method":"keepalived","params":{"id":"550562d5151bc30f"} 
data={"id":3,"jsonrpc":"2.0","error":null,"result":{"status":"KEEPALIVED"}}.

Once run the process starts communicating with the pool to get the jobs to work with.

data={"jsonrpc":"2.0","method":"job","params":{"blob":"0e0e95a7ce8806d9152b51d43091e17f30a12f09814b5d732d1d6a077db35a6a49ca1e619ee567000000921868488b28bdc8ef3a856fb46819b46f77475ca5160b0d42f95800a27e15312151","job_id":"9s2","target":"b2df0000","algo":"rx/0","height":2424652,"seed_hash":"ecd3a12c863b63d19b9558d0554ce42e9b5eeceba9c08c7da75f409f1f2980a1"}}. 

And the binary submits back the results with:

data={"id":13,"jsonrpc":"2.0","method":"submit","params":{"id":"5ccc9a1e092bd09e","job_id":"9si","nonce":"74060092","result":"744b58e43727853209fd371c84712e8f5df81e42f4fccd64a0c2a25025790000","algo":"rx/0"}}.

Syscall Inspection of the random named process

The other process was named with a random string and it executed traffic over different IPs.

From the syscall extracted, we can see the traffic generated to different public IPs using different ports.

As reported already from older versions of the binary and analyzing the syscall, we can say that this process is in charge of replicating and spreading itself horizontally on other machines trying to connect to public IPs as well as executing different exploit modules loaded in the binary.

#3 Information gathering phase and replication

The script proceeds its actions with the information gathering phase, looking for useful information and secrets.

The script gathers SSH information reading the SSH private keys and configuration files in the /, /root and /home directories.

The malware combines them with the obtained SSH keys from users and root. When the blasting is successful, download and run the ldr.sh script on the new victim host.

Summary of IOC and suspicious activities

IPs & URLs

• http[:]//194.145.227.21/ldr.ps1
• http[:]//194.145.227.21/ldr.sh
• http[:]//194.145.227.21/sys.exe
• http[:]//194.145.227.21/sys.x86_64

Files

• Ldr.sh
• ldr.ps1
• Sys.x86_64
• sys.exe

Suspicious activities

A few suspicious activities worth mentioning:

• wget or curl is launched, especially inside a container during run time (not build time).
• Installing and uninstalling packages and monitoring products.
• A network communication with the attacker machine, and anomaly outbound traffic over the internet.
• A CPU usage surge due to an unknown process launched.

Detecting crypto miners and botnets with Falco

The detection of this specific crypto miner botnet can be done in different ways and with different tools. Since crypto miners follow very distinctive patterns, we can use their behavior to create a strong detection.

One way is by using infrastructure monitoring tools, scanning the pod resources used and alerting on high CPU or GPU usage.

The other option is the “security” way, by detecting suspicious connections or malicious binary executions inside the pod.

Let’s dive into both approaches.

Falco detection

Falco is the CNCF open-source project for runtime threat detection for containers and Kubernetes.

One of the benefits of Falco is in leveraging its powerful and flexible rules language. As a result, Falco will generate security events when it finds abnormal behaviors as defined by a customizable set of rules. Meanwhile, Falco comes with a handful of out-of-the-box detection rules.

- rule: Unexpected outbound connection destination
  desc: Detect any outbound connection to a destination outside of an allowed set of ips, networks, or domain names
  condition: >
    consider_all_outbound_conns and outbound and not
    ((fd.sip in (allowed_outbound_destination_ipaddrs)) or
     (fd.snet in (allowed_outbound_destination_networks)) or
     (fd.sip.name in (allowed_outbound_destination_domains)))
  output: Disallowed outbound connection destination (command=%proc.cmdline connection=%fd.name user=%user.name user_loginuid=%user.loginuid container_id=%container.id image=%container.image.repository)
  priority: NOTICE
  tags: [network]
- rule: Container Drift Detected (chmod)
  desc: New executable created in a container due to chmod
  condition: >
    chmod and
    consider_all_chmods and
    container and
    not runc_writing_var_lib_docker and
    not user_known_container_drift_activities and
    evt.rawres>=0 and
    ((evt.arg.mode contains "S_IXUSR") or
    (evt.arg.mode contains "S_IXGRP") or
    (evt.arg.mode contains "S_IXOTH"))
  exceptions:
    - name: proc_name_image_suffix
      fields: [proc.name, container.image.repository]
      comps: [in, endswith]
    - name: cmdline_file
      fields: [proc.cmdline, fd.name]
      comps: [in, in]
      values:
        - [["runc:[1:CHILD] init"], [/exec.fifo]]
  output: Drift detected (chmod), new executable created in a container (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline filename=%evt.arg.filename name=%evt.arg.name mode=%evt.arg.mode event=%evt.type)
  priority: ERROR
- rule: Outbound Connection to C2 Servers
  desc: Detect outbound connection to command & control servers
  condition: outbound and fd.sip in (c2_server_ip_list)
  exceptions:
    - name: proc_proto_sport
      fields: [proc.name, fd.l4proto, fd.sport]
  output: Outbound connection to C2 server (command=%proc.cmdline connection=%fd.name user=%user.name user_loginuid=%user.loginuid container_id=%container.id image=%container.image.repository)
  priority: WARNING
  tags: [network]
- rule: Write below etc
  desc: an attempt to write to any file below /etc
  condition: write_etc_common
  output: "File below /etc opened for writing (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname pcmdline=%proc.pcmdline file=%fd.name program=%proc.name gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4] container_id=%container.id image=%container.image.repository)"
  priority: ERROR
  tags: [filesystem, mitre_persistence]

You can check the full rule descriptions on GitHub.

The reported Falco rules can be very useful to detect malicious actions performed by the initial script and the binary launched, but also for attacks similar to the one reported in this article.

In particular, by customizing those rules with the IoCs reported in this article, it’s possible to create a very strong detection for this specific botnet and get alerted in case something bad happens.

Resource Monitoring

Since the main adversary’s goal is to run a crypto miner in your infrastructure and use your resources for cryptocurrency, it’s possible to use any monitoring tools to check and get alerted in case something is using a high percentage of cpu inside your infrastructure.

The following PromQL query will provide the CPU usage for each workload:

sum (rate (container_cpu_usage_seconds_total{container=~".+"}[1w])) 
by (container) /  ignoring (container) group_left 
sum( machine_cpu_cores{}) * 100

If you set up proper limits and requests, affected containers will be CPU throttled. You can find those containers with the following PromQL query:

(sum by (namespace,pod,container)(rate(container_cpu_usage_seconds_total{container!=""}[5m])) / sum by (namespace,pod,container)(kube_pod_container_resource_limits{resource="cpu"})) > 0.8

Conclusion

This incident confirms a trend of crypto mining attacks being on the rise, and they are getting more creative over time.

As a system administrator, you must use the proper tools to detect these attacks. Without deep insight into the process activities, file activities, and network activities from your cloud native environment, and the help from a smart detection engine, it will be hard to detect such an attack. It will be even more difficult to uncover it.

It is also important to note that unified security and monitoring solutions will speed up the investigation process. Once you identify a single suspicious event, it helps you trace down the event from different angles, such as resource usage, network connections, and reading sensitive files.

If you would like to find out more about Falco:

• Get started at Falco.org.
• Check out the Falco project on GitHub.
• Get involved with the Falco community.
• Meet the maintainers on the Falco Slack.
• Follow @falco_org on Twitter.

---

Detecting with Sysdig Secure

The Sysdig Secure DevOps Platform is built on top of Falco and was used to detect this particular attack.

With the help of Sysdig Secure image profiling, DevOps can:

• Create a policy to detect any port bind and listen for activities from random processes.
• Create a policy to detect any destination IPs which are not in the white list.
• Create a policy to detect any processes and scripts launched which are not in the allow list (e.g., wget,curl,ldr.sh, etc.).

At Sysdig Secure, we extend Falco with out-of-the-box rules along with other open source projects, making it even easier to work with and manage Kubernetes security. Register for our Free 30-day trial and see for yourself!

The Sysdig Secure DevOps Platform provides security to confidently run containers, Kubernetes, and cloud services. With Sysdig, you can secure the build pipeline, detect and respond to runtime threats, continuously validate compliance, and monitor and troubleshoot cloud infrastructure and services. Try it today!