Blog Icon

Blog Post

File Integrity Monitoring: Detecting suspicious file activity inside a container

In this blog, we will explore suspicious file activity inside a container and see how to effectively implement a file integrity monitoring (FIM) workflow. We’ll also cover how Sysdig Secure can help you implement FIM for both containers and Linux hosts.

What is file integrity monitoring (FIM)?

File integrity monitoring gives you visibility into all of your sensitive file related activity. It’s used to detect tampering of critical system files, directories and unauthorized changes, regardless of whether the activity is a malicious attack or an unplanned operational activity.

Why is file integrity monitoring (FIM) important in container security?

Knowing if files have been tampered with is critical to keep your infrastructure secure, helping you detect attacks as soon as possible and investigate them afterwards.

For compliance: FIM is a core requirement to meet many regulatory container compliance standards like PCI-DSS, NIST, SOC2, HIPAA and more, as well as security best practice frameworks like the CIS benchmarks.

For incident response and forensics: When an unauthorized event occurs, you’ll need a full audit trail that describes how sensitive files were modified and who made those changes. This is challenging, given that 52% of containers live for five minutes or less

Attack scenario: Copying malware into a file directory

Let’s see file integrity monitoring in action with an example.

We’ll go through an attack where a hacker is copying malware to a sensitive file inside a container. Then, we’ll see how a robust FIM solution can detect and analyze all activity surrounding this attack, helping you respond effectively.

Attack scenario for file integrity monitoring, an attacker copying malware into a container. We want to receive an alert, and gather forensics data to investigate what happened.

Suppose a hacker gets access to your container (can also be a VM or a bare metal host) and starts to copy malware into /usr/bin directory.

Terminal showing an attacker copying malware into a container
The modification or attempt to modify a file path inside a container is an indicator of compromise.

If you were monitoring your host/container at runtime, you would have been alerted to this suspicious activity.

A suspicious file change alert in sysdig secure
Suspicious file change rule was immediately triggered and the cp malware /user/bin/dpkg command.

Let’s go a step further to drill into the activity surrounding this file change. This attack was performed by a user named JohnDoe who kube-exec’ed into a pod and copied malware into a directory that was part of a sensitive java application namespace.

A suspicious file change alert in Sysdig Secure audit log, showing what files were changed
Filter your audit trail by kube-exec sessions, connections made and specific file attributes (filename, directory, commands, permissions).

It’s hard to know the details of what’s happening inside Kubernetes at any given time, especially since containers are ephemeral. But this audit trail shows exactly how a file was tampered with and who made the change, all within the context of cloud and Kubernetes environments and even after the container is gone.

Best practices for FIM for hosts and containers

Four best practices for file integrity monitoring, set image scanning policies, runtime policies, automated response, and gather forensics data.

File Integrity Monitoring needs to be addressed across the container lifecycle. We will cover four best practices that you can implement in your container and Kubernetes environments.

1. Bake FIM checks into your image scanning policy

Scan for specific file attributes and embed them as part of the image scanning policy within your CI/CD pipelines. This allows you to shift security left and fail builds early if FIM policies are not met.

Start with identifying file attributes such as:

  • Check if a file exists or is missing, and trigger alerts based on the condition.
  • Validate a specific file against its SHA256 hash. You already know the SHA256 for the binary or binaries in your containers. Any modification to the executable files is suspicious and potentially dangerous.
  • Validate file permissions. For example, if a file has an executable bit where it’s not expected, you should flag that as an alert.
  • Check for file names based on regex.
  • Inspect contents, looking for exposed passwords, credential leaks, etc.
A file sha256 check in sysdig secure image scanning policies
Sysdig Secure provides default container image scanning policies and user defined policies.

2. Create Runtime Policies to monitor for Filesystem Changes

Implement detection policies at runtime that would alert on any suspicious changes to a filesystem. These are common file integrity monitoring checks that you should include as rules to enforce a strong security posture:

  • Creation or removal of files or directories.
  • Renaming of files or directories.
  • Changes to file or directory security settings such as permissions, ownership and inheritance.
  • Changes to the files of a container.
  • Modification of files below the container’s path.
  • Deletion of bash history.
An extensive set of out of the box runtime detection rules in sysdig secure.
Filesystem policies in the Rules Library in Sysdig Secure. This makes it easy for you to quickly implement FIM policies.

3. Implement an automated response mechanism

In the case of a file tampering event, automate remediation responses such as:

  • Notify when a violation occurs via Slack, SNS, JIRA, email, PagerDuty, etc.
  • Pause and quarantine the container.
  • Kill the container to stop the attack.
Automated remediation for file integrity monitoring in sysdig secure
Sysdig Secure can automatically remediate and trigger notifications to your alerting tools.

4. Ensure you have comprehensive forensics data

Typically, attackers use clever tactics that have been successful in evading traditional security and forensics tools. To combat this, gathering low level syscall data, that is enriched with container/Kubernetes metadata, gives you the single source of truth that can’t be fooled.

Use open-source Sysdig Inspect to analyze everything that happened right before and after a suspicious file activity.

forensics for file integrity monitoring with sysdig inspect.
Sysdig Secure forensics capabilities (Captures) can record all of the pre- and post-attack container activity – evidence that an attacker has tampered with a host in system files and configurations.

Try Sysdig Secure for file integrity monitoring

Sysdig Secure can support you in managing security risk by helping you implement FIM for hosts and containers. To learn more, visit https://sysdig.com/products/kubernetes-security/ and sign up for a 30 day free Sysdig Secure trial!

Share This

Stay up to date

Sign up to receive our newest.

Related Posts

PCI Compliance for Containers and Kubernetes

Five things CISOs can do to make containers secure and compliant

Sysdig Extends Security Across All AWS Container Services