In this blog, we will explore suspicious file activity inside a container and see how to effectively implement a file integrity monitoring (FIM) workflow. We’ll also cover how Sysdig Secure can help you implement FIM for both containers and Linux hosts.
What is file integrity monitoring (FIM)?
File integrity monitoring gives you visibility into all of your sensitive file related activity. It’s used to detect tampering of critical system files, directories and unauthorized changes, regardless of whether the activity is a malicious attack or an unplanned operational activity.
Why is file integrity monitoring (FIM) important in container security?
Knowing if files have been tampered with is critical to keep your infrastructure secure, helping you detect attacks as soon as possible and investigate them afterwards.
For compliance: FIM is a core requirement to meet many regulatory container compliance standards like PCI-DSS, NIST, SOC2, HIPAA and more, as well as security best practice frameworks like the CIS benchmarks.
For incident response and forensics: When an unauthorized event occurs, you’ll need a full audit trail that describes how sensitive files were modified and who made those changes. This is challenging, given that 52% of containers live for five minutes or less
📄👀 Malware, leaked credentials or attacks can be detected with file integrity monitoring.Learn how with these 4 best practices.Click to tweet
Attack scenario: Copying malware into a file directory
Let’s see file integrity monitoring in action with an example.
We’ll go through an attack where a hacker is copying malware to a sensitive file inside a container. Then, we’ll see how a robust FIM solution can detect and analyze all activity surrounding this attack, helping you respond effectively.
Suppose a hacker gets access to your container (can also be a VM or a bare metal host) and starts to copy malware into /usr/bin directory.
If you were monitoring your host/container at runtime, you would have been alerted to this suspicious activity.
Let’s go a step further to drill into the activity surrounding this file change. This attack was performed by a user named JohnDoe who kube-exec’ed into a pod and copied malware into a directory that was part of a sensitive java application namespace.
It’s hard to know the details of what’s happening inside Kubernetes at any given time, especially since containers are ephemeral. But this audit trail shows exactly how a file was tampered with and who made the change, all within the context of cloud and Kubernetes environments and even after the container is gone.
Best practices for FIM for hosts and containers
File Integrity Monitoring needs to be addressed across the container lifecycle. We will cover four best practices that you can implement in your container and Kubernetes environments.
1. Bake FIM checks into your image scanning policy
Scan for specific file attributes and embed them as part of the image scanning policy within your CI/CD pipelines. This allows you to shift security left and fail builds early if FIM policies are not met.
Start with identifying file attributes such as:
- Check if a file exists or is missing, and trigger alerts based on the condition.
- Validate a specific file against its SHA256 hash. You already know the SHA256 for the binary or binaries in your containers. Any modification to the executable files is suspicious and potentially dangerous.
- Validate file permissions. For example, if a file has an executable bit where it’s not expected, you should flag that as an alert.
- Check for file names based on regex.
- Inspect contents, looking for exposed passwords, credential leaks, etc.
2. Create Runtime Policies to monitor for Filesystem Changes
Implement detection policies at runtime that would alert on any suspicious changes to a filesystem. These are common file integrity monitoring checks that you should include as rules to enforce a strong security posture:
- Creation or removal of files or directories.
- Renaming of files or directories.
- Changes to file or directory security settings such as permissions, ownership and inheritance.
- Changes to the files of a container.
- Modification of files below the container’s path.
- Deletion of bash history.
3. Implement an automated response mechanism
In the case of a file tampering event, automate remediation responses such as:
- Notify when a violation occurs via Slack, SNS, JIRA, email, PagerDuty, etc.
- Pause and quarantine the container.
- Kill the container to stop the attack.
4. Ensure you have comprehensive forensics data
Typically, attackers use clever tactics that have been successful in evading traditional security and forensics tools. To combat this, gathering low level syscall data, that is enriched with container/Kubernetes metadata, gives you the single source of truth that can’t be fooled.
Use open-source Sysdig Inspect to analyze everything that happened right before and after a suspicious file activity.
Try Sysdig Secure for file integrity monitoring
Sysdig Secure can support you in managing security risk by helping you implement FIM for hosts and containers. To learn more, visit https://sysdig.com/products/kubernetes-security/ and sign up for a 30 day free Sysdig Secure trial!