Terraform Container Security as Code with Sysdig Secure

In this blog post, you will learn how to use the Sysdig Terraform provider to manage and automate security configurations for your infrastructure. A Terraform provider is essentially a plugin for Terraform that contains all the logic needed to interact with various APIs and create the infrastructure or configurations you define as code using the HCL language.

With this provider, you will be able to:

Version-Control Your Settings: Store and manage Sysdig configurations in a Git repository. This lets your teams collaborate on changes, review updates, and track revisions. You can also roll back easily to previous versions when needed.

Ensure Consistency Across Environments: Use Terraform to replicate configurations across development, staging, and production environments, ensuring consistency and avoiding drift. Even when managing multiple environments, a single provider block can be reused by dynamically passing environment-specific variables, simplifying your overall configuration.
Automate Repetitive Tasks: Save time by automating the creation of alerts, policies, and other configurations.
Unified Workflow: Manage both Sysdig Secure and Monitor resources using a single tool. For instance, you can create runtime policies in Sysdig Secure and configure monitoring alerts in Sysdig Monitor within the same Terraform configuration. This reduces complexity and improves workflow efficiency.

Installing the Sysdig Terraform Provider

To get started, ensure Terraform version 0.13 or higher is installed. If you’re new to Terraform, create a main.tf file with the following contents and initialize your project using terraform init.

terraform {
  required_providers {
    sysdig = {
      source = "sysdiglabs/sysdig"
      version = ">=1.46.0"
    }
  }
}Code language: Perl (perl)

This will set up your environment and download the provider, as it is available on the Terraform Registry.

Configuring the Sysdig Provider

The Sysdig provider supports resources and data sources for both Sysdig Secure and Sysdig Monitor. Sysdig Secure focuses on runtime security, compliance, and vulnerability management, while Sysdig Monitor provides advanced monitoring, alerting, and the ability to analyze all your metrics in dashboards.

Each service requires its own API token and endpoint, allowing users to address distinct use cases with targeted configurations.

Secure authentication

provider "sysdig" {
  sysdig_secure_url      = "https://secure.sysdig.com"
  sysdig_secure_api_token = "<your_secure_api_token>"
}Code language: Perl (perl)

Monitor authentication

provider "sysdig" {
  sysdig_monitor_url     = "https://app.sysdigcloud.com"
  sysdig_monitor_api_token = "<your_monitor_api_token>"
}Code language: Perl (perl)

Combined Secure and Monitor authentication

You can also combine Secure and Monitor configurations within a single provider block. This approach simplifies management by allowing users to define Secure and Monitor settings in one place. It also supports environments with proxies by enabling the inclusion of extra HTTP headers through the extra_headers block.

provider "sysdig" {
  sysdig_secure_url       = "https://secure.sysdig.com"
  sysdig_secure_api_token = "<your_secure_api_token>"

  sysdig_monitor_url      = "https://app.sysdigcloud.com"
  sysdig_monitor_api_token = "<your_monitor_api_token>"

  extra_headers = {
    "Proxy-Authorization" = "Basic xxxxxxxxxxxxxxxx"
  }
}Code language: Perl (perl)

Supported resources and data sources

The Sysdig Terraform provider supports a variety of resources and data sources for Sysdig Platform, Sysdig Secure, and Sysdig Monitor. Here’s a quick overview:

Sysdig Platform (both Monitor and Secure)

User and Access Management: Manage users, roles, and service accounts (e.g., sysdig_user, sysdig_custom_role).
IP Filtering: Configure IP restrictions for platform access (e.g., sysdig_ip_filter).

Sysdig Secure

Policies: Create runtime and vulnerability policies (e.g., sysdig_secure_policy).
Compliance controls: Define posture controls (e.g., sysdig_secure_posture_control).
Rules: Implement runtime rules using Falco syntax (e.g., sysdig_secure_rule).
Notification channels: Set up alerts for email, Slack, and other integrations (e.g., sysdig_secure_notification_channel_slack).
Macros and lists: Reuse logic across multiple Falco rules (e.g., sysdig_secure_macro, sysdig_secure_list).

Sysdig Monitor

Alerts: Create alerts triggered by metrics, PromQL queries, or anomalies detected from abnormal behavior (e.g., sysdig_monitor_alert_metric).
Dashboards and teams: Manage dashboards and user teams through code, configuring display preferences as code (e.g., sysdig_monitor_dashboard).
Notification channels: Configure integrations for alerts, such as PagerDuty, webhooks, or email (e.g., sysdig_monitor_notification_channel_email).

Advanced use cases

Automating configuration replication

Organizations often use multiple environments, such as development, staging, and production. The Sysdig Terraform provider helps replicate and test configurations across these environments efficiently. For instance:

Custom Falco Rules: Test runtime detection rules in staging to validate them before promoting to production, reducing disruptions and noise.
Compliance Policies: Use Terraform to roll out Rego-based policies incrementally, refining and validating them across environments.

Terraform uses provider blocks and state files to ensure each environment is provisioned with consistent configurations, avoiding drift. Its declarative model allows you to define the desired end state, and Terraform handles the underlying steps to match that state, this reduces manual errors and improves efficiency in the process. Plus, since the required state is defined as code, you can save it in a version control system like Git to track all the past changes, review new changes and roll them back if needed.

Debugging

For troubleshooting configurations or debugging Terraform runs, you can enable detailed logging by setting the TF_LOG=DEBUG environment variable. This will display API calls and other details, helping you identify any issues.

Tutorial: Creation of a policy as code from scratch

Now, let’s put ourselves in the shoes of someone who wants to configure their environment with Terraform and the provider. First, we will start creating a main.tf file with the configuration of the provider.

terraform {
  required_providers {
    sysdig = {
      source = "sysdiglabs/sysdig"
      version = ">=1.46.0"
    }
  }
}

provider "sysdig" {
  sysdig_secure_url      = "https://secure.sysdig.com"
  sysdig_secure_api_token = "<your_secure_api_token>"
}Code language: Perl (perl)

After executing terraform init you will see the following message:

Initializing the backend...

Initializing provider plugins...

- Finding sysdiglabs/sysdig versions matching "1.46.0"...

- Installing sysdiglabs/sysdig v1.46.0...

- Installed sysdiglabs/sysdig v1.46.0 (signed by a HashiCorp partner, key ID 0A2458C5D4F39147)

Partner and community providers are signed by their developers.

If you'd like to know more about provider signing, you can read about it here:

https://www.terraform.io/docs/cli/plugins/signing.html

Terraform has created a lock file .terraform.lock.hcl to record the provider

selections it made above. Include this file in your version control repository

so that Terraform can guarantee to make the same selections by default when

you run "terraform init" in the future.

Terraform has been successfully initialized!
Code language: Perl (perl)

The policy

We want to detect the execution of a terminal shell within a container in our organization. This event is usually a security risk, as spawning shells in production containers can indicate unauthorized access, misconfigurations, or attempts to exploit the container environment.

We can start with the creation of the policy:

resource "sysdig_secure_custom_policy" "terminal_shell_in_container" {
  depends_on = [sysdig_secure_rule_falco.terminal_shell_in_container]
  
  name = "Terminal shell in container"
  description = "Detected a terminal shell being open within a container"
  severity = 4
  enabled = true

  actions {
    capture {
      name = "terminal-shell.scap"
      seconds_before_event = 5
      seconds_after_event = 10
    }
  }
}Code language: Perl (perl)

We have defined the overall policy, but policies are only bundles of rules with added behavior, like in this case, we are capturing all the syscalls surrounding the event. On its own, this won’t detect anything, so we need to feed it with a rule (or a bunch of rules).

The rule

Following the Falco rule syntax, we can create custom rules in Sysdig Secure with Terraform. Falco enables precise detection of suspicious activities by leveraging system call data. In this case, we will create a custom rule to detect terminal shells being opened within containers. You don’t need to understand the syntax right now, Sysdig already provides more than 1400 rules so you can always take a look at the default ones and create your own.

resource "sysdig_secure_rule_falco" "terminal_shell_in_container" {
  name        = "Terminal shell in container" // ID
  description = "A shell was used as the entrypoint/exec point into a container with an attached terminal."
  tags        = ["container", "shell"]

  condition = "(evt.type in (execve, execveat) and evt.dir=<) and (container.id != host) and (proc.name in (ash, bash, csh, ksh, sh, tcsh, zsh, dash))"
  output    = "A shell was spawned in a container with an attached terminal (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository)"
  priority  = "notice"
  source    = "syscall"
}Code language: Perl (perl)

As you can see, the rule’s condition specifies multiple Falco field events, which may not be as readable as we would like. Additionally, whenever we need to modify it, we must reinterpret its logic. Furthermore, the shells we are detecting are hardcoded in the rule, which is another area for improvement.

This is where Falco’s concepts of lists and macros come in.

Extracting the list of shells

We can enhance the rule by extracting the list of shells we want to detect. This improves maintainability by centralizing the list, making it easier to update in the future without modifying the rule directly.

resource "sysdig_secure_list" "shell_binaries" {
  name = "shell_binaries"
  items = ["ash", "bash", "csh", "ksh", "sh", "tcsh", "zsh", "dash"]
}

resource "sysdig_secure_rule_falco" "terminal_shell_in_container" {
  depends_on = [sysdig_secure_list.shell_binaries]
  
  name        = "Terminal shell in container" // ID
  description = "A shell was used as the entrypoint/exec point into a container with an attached terminal."
  tags        = ["container", "shell"]

  condition = "(evt.type in (execve, execveat) and evt.dir=<) and (container.id != host) and (proc.name in (${sysdig_secure_list.shell_binaries.name}))"
  output    = "A shell was spawned in a container with an attached terminal (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository)"
  priority  = "notice"
  source    = "syscall"
}Code language: Perl (perl)

As you can see, we have updated the rule’s condition to reference the previously created list, making the rule dependent on it through the depends_on statement.

We can further simplify the condition by extracting macros and referencing them.

Extracting macros

resource "sysdig_secure_list" "shell_binaries" {
  name = "shell_binaries"
  items = ["ash", "bash", "csh", "ksh", "sh", "tcsh", "zsh", "dash"]
}

resource "sysdig_secure_macro" "shell_procs" {
  depends_on = [sysdig_secure_list.shell_binaries]
  
  name = "shell_procs"
  condition = "(proc.name in (${sysdig_secure_list.shell_binaries.name})"
}

resource "sysdig_secure_macro" "spawned_process" {
  name = "spawned_process"
  condition = "(evt.type in (execve, execveat) and evt.dir=<)"
}

resource "sysdig_secure_macro" "container" {
  name = "container"
  condition = "(container.id != host)"
}

resource "sysdig_secure_rule_falco" "terminal_shell_in_container" {
  depends_on = [sysdig_secure_macro.shell_procs, sysdig_secure_macro.spawned_process, sysdig_secure_macro.container]
  
  name        = "Terminal shell in container" // ID
  description = "A shell was used as the entrypoint/exec point into a container with an attached terminal."
  tags        = ["container", "shell"]

  condition = "(${sysdig_secure_macro.shell_procs.name} and ${sysdig_secure_macro.spawned_process.name} and ${sysdig_secure_macro.container.name})"
  output    = "A shell was spawned in a container with an attached terminal (user=%user.name %container.info shell=%proc.name parent=%proc.pname cmdline=%proc.cmdline terminal=%proc.tty container_id=%container.id image=%container.image.repository)"
  priority  = "notice"
  source    = "syscall"
}

resource "sysdig_secure_custom_policy" "terminal_shell_in_container" {
  depends_on = [sysdig_secure_rule_falco.terminal_shell_in_container]
  
  name = "Terminal shell in container"
  description = "Detected a terminal shell being open within a container"
  severity = 4
  enabled = true

  rules {
    name = sysdig_secure_rule_falco.terminal_shell_in_container.name
    enabled = true
  }

  actions {
    capture {
      name = "terminal-shell.scap"
      seconds_before_event = 5
      seconds_after_event = 10
    }
  }
}Code language: Perl (perl)

Now, we have the final code ready to apply! Notice how the macros enhance visibility in the conditions—by naming them, we can clearly identify their purpose. This approach simplifies complex conditions and makes updates or debugging much more manageable.

Next, we can apply these resources using Terraform with terraform apply.

Plan: 6 to add, 0 to change, 0 to destroy.
sysdig_secure_macro.spawned_process: Creating...
sysdig_secure_macro.container: Creating...
sysdig_secure_list.shell_binaries: Creating...
sysdig_secure_macro.spawned_process: Creation complete after 9s [id=26732344]
sysdig_secure_list.shell_binaries: Still creating... [10s elapsed]
sysdig_secure_macro.container: Still creating... [10s elapsed]
sysdig_secure_list.shell_binaries: Creation complete after 16s [id=26732345]
sysdig_secure_macro.shell_procs: Creating...
sysdig_secure_macro.container: Still creating... [20s elapsed]
sysdig_secure_macro.container: Creation complete after 23s [id=26732346]
sysdig_secure_macro.shell_procs: Still creating... [10s elapsed]
sysdig_secure_macro.shell_procs: Creation complete after 20s [id=26732347]
sysdig_secure_rule_falco.terminal_shell_in_container: Creating...
sysdig_secure_rule_falco.terminal_shell_in_container: Creation complete after 7s [id=26732348]
sysdig_secure_custom_policy.terminal_shell_in_container: Creating...
sysdig_secure_custom_policy.terminal_shell_in_container: Creation complete after 0s [id=10410832]

Apply complete! Resources: 6 added, 0 changed, 0 destroyed.Code language: Perl (perl)

We can see the result in Sysdig Secure – the policy with the attached rule has been created.

This will be triggered as soon as Sysdig detects that a terminal was opened in a container, like in this example:

Conclusion

The Sysdig Terraform provider simplifies managing and automating security configurations, ensuring consistency and compliance across environments. For detailed examples, advanced configurations, and a full list of supported resources and data sources, explore the Sysdig Terraform Provider documentation.

Using Terraform for container security as code with Sysdig