Protection From Malicious Python Libraries Jeilyfish and Python3-dateutil

Two malicious Python libraries, jeilyfish (with a capital i and a lowercase L in the original name) and python3-dateutil, were detected on PyPI (Python Package Index) on December 1st. They were typosquatting similar named legitimate libraries jellyfish (with a double lowercase L) and python-dateutil libraries, a malicious technique aiming to trick developers to use the similar named modified libraries.

In this article we will explain their behavior and how you can use Sysdig Secure to detect and stop them in the CI/CD pipeline using image scanning and runtime policies with Falco rules.

Two malicious Python libraries were found on PyPI, jeilyfish and python3-dateutil. Learn how destructive they can be and how to protect against them. Click to tweet

A year with a malicious typosquatting library undetected in the official PyPI index.

Both trojanized libraries based their code on the legitimates ones, so they were a perfect replacement for them, not giving many hints about the security concern.

From the same developer, they were promptly removed from the index on the same day they were reported. The trojanized jeilyfish library had been on the index for almost a whole year (December 11 of 2018), while python3-dateutil had only been there for a couple of days. Jeilyfish recounted more than 140 downloads only from the last month it was available.

Jeilyfish, stealing SSH and GPG keys

Jeilyfish library is the nastier of the malicious Python libraries. The library included a base64 compressed text that was, in fact, Python code. When decoded, uncompressed and executed, it would connect to a URL to download another bunch of encoded code, that again when decoded and executed copies the SSH and GPG keys of the home directory of the current user, and uploads them to a server with HTTP address http://68.183.212.246:32258 (it has already been blocked by its hosting provider). It also compiles a list of information about the contents of several directories (home, Documents, Downloads, PycharmProjects), and information about the IP address of the trojanized computer and its geolocation data using the ifconfig.co service.

The directories from which it would list contents would include PycharmProjects, used by the Pycharm editor for Python, so it would gain knowledge about Python projects that may be published using those SSH and GPG keys, to tamper with them. Any developer that detects this library in any of their projects must invalidate their keys as soon as possible, and review any recent changes to projects using those keys.

Python3-dateutil, malicious by reference

The python3-dateutil library is the easiest to explain. From the same developer, it would just reference jeilyfish, so it is considered malicious by reference. We have to consider also that some distributions like Fedora uses the “python3-dateutil” name for the package as a legitimate one (but not PyPI), so if we try blocking it referenced by its typosquatting name, we could end up with a false positive. Because of that, we will focus on blocking the jeilyfish library, as that will detect and prevent any malicious activity.

Protecting from malicious Python libraries with Sysdig Secure

We are going to describe steps to use Sysdig Secure to detect and stop the jeilyfish both at the CI/CD pipeline and at runtime. We will also explain how you can detect some of its malicious behavior at runtime, so even if a modified version of the threat is not detected in the pipeline, you are protected from its effects. The same steps could be taken to protect from any other similar zero-day threat and typosquatting libraries that must be addressed as soon as possible.

Detecting malicious Python libraries in the CI/CD pipeline

The best way to prevent a security threat is to stop it from even reaching the production environment. You can use Sysdig Secure to integrate image scanning with your CI/CD pipeline development tool to prevent threats at the development level. You can integrate with Jenkins CI/CD, GitHub Actions, Gitlab CI/CD, Azure Pipelines, Atlassian Bamboo, AWS CodePipeline and AWS CodeBuild among others.

As an extra, also remember that using these rules, you can trigger manual scans of images in the registry, scans of running containers, and even scan local images without access to the registry.

Creating a policy for image scanning of typosquatting malicious Python package

First, you will start creating a policy for all rules useful to detect the usage of the malicious Python package. Go to your Sysdig Secure dashboard, and navigate to the image scanning policies list using the menu items Image Scanning > Image scanning policies > Policies, and click on the Add policy button.

• Name: Jeilyfish malicious Python library (typosquatting)
• Description: Usage of jeilyfish (with a capital i before the lowercase L) typosquatting malicious Python library, supplanting the real jellyfish library (with two Ls) from PyPI.

For the Rules section, you will add several of them, starting always by selecting a gate.

Image scanning rule to detect typosquatting malicious Python package

The first rule we are going to explain checks if a Python package with a given name is used in a container image. Enter this data in the form:

• Gate: Packages
• Blacklist
• Select parameters (click to expand parameter list)
  • Name: jeIlyfish
  • Version (optional): (leave blank)
• Stop

(notice the name has a capital i before the lowercase L).

Image scanning rule to detect malicious Python libraries install attempts

Now you will create a rule to detect when the Dockerfile for building a container image references in some way the offending library. Select a new Gate to start.

• Gate: Dockerfile
• Instruction
• Check (click to expand parameter list)
  • Actual dockerfile only: Leave blank
  • Check: like
  • Instruction: RUN
  • Value: .*jeIlyfish.*
• Stop

(notice the value input has a capital i before the lowercase L).
</p

Image scanning rule to detect a malicious file name

This policy will detect the presence of the directory and file name of the malicious file. You could also add the checksum of a file if you know it, but in this case, the package has quickly been removed. Let’s assume all we know is a directory name, and the file it contains: jeIlyfish/_jellyfish.py (characters after “_je” are a capital i and a lowercase L). You will use that information to detect it in container images with the following policy.

Click add new policy, and use these values:

• Gate: Files
• Name match
• Regex: .*jeIlyfish\/_jellyfish\.py
• Stop

(notice the directory name has a capital i before the lowercase L).

After adding the three rules, your policy will look like this:

Click Save to finally add the policy for image scanning.

Detecting malicious activity at runtime with Falco

We have shown you how to scan container images, but you may be monitoring a physical host, or a development cluster, where developers build the images locally without running scans and push them to execution without a registry. We also want to show you how to detect the symptoms of malicious activity, so you can create derived rules for different new threats.

Now we will show you how to define Falco runtime rules in Sysdig Secure, that are checked at execution time. Falco is a Kubernetes runtime security tool that you can use for behavioral activity monitoring.

Falco rule to detect outgoing connections to a malicious IP address (68.183.212.246)

The first rule we are going to show you is going to detect connections to the jeilyfish malicious Python library server IP address. This is something that can only be detected at runtime, and you could customize this rule for other similar threats once you know which IP address you want to detect connections to.

To create this rule, navigate in Sysdig Secure dashboard to Policies > Rules > Rules Library and click Add Rule

• Name: Connection to jeilyfish malicious Python library server at 68.183.212.246
• Description: Detect attempts to connect to a malicious server (68.183.212.246)
• Condition:
  outbound and fd.sip="68.183.212.246"
• Output:
  Outbound connection to jeilyfish malicious Python library server (68.183.212.246) (command=%proc.cmdline connection=%fd.name %container.info image=%container.image)
• Priority: Critical
• Source: Syscall
• Tags: network, jeilyfish

The code that describes this Falco rule:

- rule: Connection to Jeilyfish malicious Python library server
  desc: Detect attempts to connect to a malicious server (68.183.212.246)
  condition: outbound and fd.sip="68.183.212.246" 
  output: Outbound connection to Jeilyfish malicious Python library server (68.183.212.246) (command=%proc.cmdline connection=%fd.name %container.info image=%container.image)
  priority: CRITICAL
  tags: [network, python, jeilyfish, typosquatting]Code language: JavaScript (javascript)

Falco rule to prevent the installation of malicious libraries using pip

You can also detect installation attempts of these malicious python libraries directly on running containers or in any host.

Navigate again to Policies > Rules > Rules Library and click Add Rule.

• Name: Prevent installation of malicious jeilyfish Python library
• Description: Detects the execution during runtime using pip of the jeilyfish malicious Python library
• Condition:
  spawned_process and
(proc.name="pip" or proc.name="pip3") and
(proc.cmdline contains "jeilyfish")

• Output:
  Detect installation of malicious jeilyfish Python library (user=%user.name command=%proc.cmdline container=%container.info)

• Priority: Critical
• Source: Syscall
• Tags: filesystem, jeilyfish

And also the equivalent Falco rule:

- rule: Prevent installation of malicious jeilyfish Python library
  desc: Detects the execution during runtime using pip of the jeilyfish malicious Python library
  condition: >
    spawned_process and
    (proc.name="pip" or proc.name="pip3") and
    (proc.cmdline contains "jeilyfish")
  output: "Detect installation of malicious jeilyfish Python library (user=%user.name command=%proc.cmdline container=%container.info)"
  priority: CRITICAL
  tags: [filesystem, python, pip, jeilyfish, typosquatting]
Code language: JavaScript (javascript)

Falco rule to detect SSH key reads

Falco already has a rule to detect SSH key reads. You can create one that adds an additional check for the /etc/ssh directory with these parameters.

• Name: Read ssh information v2
• Description: Any attempt to read files below ssh directories by non-ssh programs
• Condition:
  (consider_ssh_reads and
(open_read or open_directory) and
(user_ssh_directory or fd.name startswith /root/.ssh or fd.name startswith /etc/ssh) and
(not proc.name in (ssh_binaries)))

• Output:
  ssh-related file/directory read by non-ssh program (user=%user.name
command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline container_id=%container.id image=%container.image.repository)

• Priority: Critical
• Source: Syscall
• Tags: filesystem, jeilyfish

The Falco rule code:

- rule: Read ssh information
  desc: Any attempt to read files below ssh directories by non-ssh programs
  condition: >
    (consider_ssh_reads and
     (open_read or open_directory) and
     (user_ssh_directory or fd.name startswith /root/.ssh or fd.name startswith /etc/ssh) and
     (not proc.name in (ssh_binaries)))
  output: >
    ssh-related file/directory read by non-ssh program (user=%user.name
    command=%proc.cmdline file=%fd.name parent=%proc.pname pcmdline=%proc.pcmdline container_id=%container.id image=%container.image.repository)
  priority: CRITICAL
  tags: [filesystem, mitre_discovery, jeilyfish, typosquatting]
Code language: PHP (php)

Falco rule to detect GPG key reads

For this rule, you will first create a list for GPG binaries that may read GPG keys. You will only include one now, but you may need to add more later.

Navigate to Policies > Rules > Falco Lists and click Add List

• List name: gpg_binaries
• Items: gpg

Now you will create a rule to detect reads to files in the .gnupg directory, either when inside /root, or /home directory. To do this, navigate again to Policies > Rules > Rules Library and click Add Rule

• Name: Read GPG keys
• Description: Any attempt to read files below user gpg key directories by non-gpg programs
• Condition:
  (open_read or open_directory) and
((fd.name startswith /root/.gnupg) or
(fd.name startswith '/home' and fd.name contains '.gnupg)) and
(not proc.name in (gpg_binaries))

• Output:
  GPG key read by non-gpg program (user=%user.name command=%proc.cmdline
file=%fd.name parent=%proc.pname)
• Priority: Critical
• Source: Syscall
• Tags: filesystem, gpg, jeilyfish

Falco rule code:

- list: gpg_binaries
  Items: [gpg]
- rule: Read GPG keys
  desc: Any attempt to read files below user gpg key directories by non-gpg  programs
  condition: >
    (open_read or open_directory) and
    ((fd.name startswith /root/.gnupg) or
     (fd.name startswith '/home' and fd.name contains '.gnupg)) and
    (not proc.name in (gpg_binaries))
  output: >
    GPG key read by non-gpg program (user=%user.name command=%proc.cmdline
    file=%fd.name parent=%proc.pname)
  priority: CRITICAL
  tags: [filesystem, gpg, secret exfiltration, jeilyfish, typosquatting]
  source: syscallCode language: PHP (php)

Create a policy to stop containers running malicious Python libraries

Runtime rules need to work in conjunction with Policies to take action and notify you of the threats. Here we explain how to create a policy that will warn you by email whenever a rule triggers, capture system events that can be inspected later to investigate the activity that took place, and stop the offending containers.

To create a policy for these rules, navigate to Policies > Runtime Policies and click Add Policy

• Name: Malicious Python library jeilyfish activities prevention
• Description: Prevent runtime activities from jeilyfish malicious Python library
• Enabled: On
• Severity: High
• Scope: Custom Scope
  Everywhere (leave blank)
• Rules
  Click Import from library, and search or filter by tag the created rules, select them and click Mark for Import rules and then, Import rules
• Actions
  • Containers: Stop
  • Capture: On
    • 5 secs before, 15 secs after the event
  • Notification Channels: Email Channel

You could also add different notifications channels in your profile Settings > Notifications, or create separate policies for each one of the detections to be able to pinpoint the problem more easily.

Conclusion

Two malicious Python libraries, jeilyfish and python3-dateutil, with typosquatting names, have been detected in the official PyPI index, stealing SSH and GPG keys. One of them was undetected for a whole year.

Using Sysdig Secure you can easily create policies and rules to detect and prevent not only the use of trojanized libraries, but also their malicious behavior; protecting your infrastructure from future incarnations of the attack.