Automated Falco rule tuning

By Eric Lugo - JUNE 9, 2021
BACK TO blog

We recently released the automated Falco rule tuning feature in Sysdig Secure.

Out-of-the-box security rules are a double-edged sword. On one side, they allow you to get started right away. On the other, it can take many working hours to learn the technology, configuration, and syntax to be able to customize the rules to fit your applications.

Falco’s default security rules are no different. In an effort to cut down on maintenance when using Sysdig Secure, while easing the customization of Falco rules, we are releasing the Falco Rule Tuner.

Keep reading to discover how this feature will help your team get better security insights while reducing the noise and helping avoid alert fatigue.

Noise vs. signal

So, you got started with a security tool and you enabled all the default security policies. Nice work! Your environment is secure 😎.

However, shortly after, you are flooded with notifications and you have to go one by one, checking if they are actually a security threat 🚨. This is fine, you tell yourself. “I’m secure.”

After all, default security policies need to have a broad scope to detect a wide range of security events so they can be applicable in any scenario. However, they are so broad that many of your applications, tooling, or even normal day-to-day activities can lead to a security event.

This makes it difficult to cut through the noise as you need to check each event, digging into different names, attributes, and environments. Too many false positives can take attention from the issues that actually need it, risking that you miss an important event.

We, at Sysdig, are aware of this problem. Our out-of-the-box ruleset is so broad that right after getting Sysdig Secure enabled, you may get hundreds of security events.

Our first step to approach this problem was our great Falco rule editor.

An admin may be able to tune out a few rules after quickly learning the Falco rules language, but this is still a manual step. So, as more applications are brought online, new events happen more frequently. Our administrator is eventually overwhelmed and may start ignoring the events, or, worse, turn off the notifications.

The promised land of the artificial intelligence unicorn

One good trait of containers and microservices is that, as they are so focused on specific tasks, any suspicious behavior deviates clearly and is easily identifiable. And machine learning just shines on that area.

Machine learning brings us closer to the promised land, where security tools just work out-of-the-box. As so, many people are trying to solve the security notification noise problem with complex machine learning algorithms. Sadly, the security space isn’t there yet, but we think this is a step in the right direction.

Believing in this ideal, we introduced image profiling with machine learning long ago. And it’s a great feature that simplified securing container images even further.

However, at Sysdig, we are not shooting for the moon. We are aiming for the stars.

We want to extend the same easy to use experience to the whole cloud infrastructure, not only container images.

The Falco Rule Tuner to the rescue

Since we launched the Sysdig Cloud Connector in late 2020, you can use Falco rules to secure both your container runtime and your cloud infrastructure. So, we’ve been working on an approach that will help you tune your security events in both environments.

The Falco Rule Tuner gives security teams a tool to automatically tune out noisy policies, default or custom.

After an administrator turns this feature on, a recommended set of “exceptions” will be added at least once per day. Then, security teams can choose whether to keep or delete the suggested exceptions permanently.

There are two things we want to highlight about the Falco Rule Tuner.

First, it is an automated analysis of rules. And let us reiterate that it is done at least once every day.

Second, we’re proud of how these exceptions were modeled.

Before we released this feature, exceptions created with the rule editor were appended to the rule condition. For that, we used an and not macro_name at the end of a rule. This was essentially just a condition that matches a list of objects we wanted to ignore. For example, we may want to ignore a container that mounts the docker daemon when running a Jenkins job.

- macro: docker_proc
  condition: proc.name="docker"
- macro: nginx_proc
  condition: proc.name="nginx"
- rule: Always True
  condition: always_true and not docker_proc and not nginx_proc

We wanted to make exceptions a first class citizen within Falco. So, as part of this tuning work, we have pushed an exception field to the upstream OSS Falco project. The previous rule would now be:

- rule: Always True
  condition: always_true
  exceptions:
  - name: docker_proc
    fields: proc.name
    comps: in
    values: [docker, nginx]

This way, the whole Falco community can benefit from a part of our new feature.

Conclusion

Noisy policies are a problem for security. They can cause alert fatigue and make you miss important security events.

Manually tweaking those policies is a time consuming task, and is a non-viable practice once your workloads grow beyond a point.

Automated and machine learning approaches bring us closer to the promised land, where security tools work out-of-the-box. However, they still need humans behind the wheels to make a final decision.

The newly introduced Falco Rule Tuner is in a sweet spot, automatically identifying noisy policies and providing an easy button to tune them.

Finally, this new feature comes with an upgrade to the Falco rule syntax, available to all of the Falco community.

If you would like to find out more about Falco:

At Sysdig Secure, we extend Falco with out-of-the-box rules, along with other open source projects, making them even easier to work with and manage Kubernetes security. Register for our Free 30-day trial and see for yourself!