What’s new in Sysdig - October 2021

Welcome to another monthly update on what’s new from Sysdig! Happy Halloween! Happy Indigenous Peoples’ day! Happy Eid Milad-Un-Nabi!

सर्व मंगल मांगल्ये शिवे सर्वार्थ साधिके |

शरण्ये त्रयम्बके गौरी नारायणी नमोस्तुते ||

This is a big month in the cloud native world as KubeCon finally returned, although as a hybrid event. It has still been great seeing many of you in person once again! For those of you that couldn’t make it, we hope you dropped by the virtual booth to say hi and watch one of our virtual office hours sessions. If you didn’t attend or missed some talks, look out for the CNCF YouTube channel for the recordings. If they aren’t there already, they will be there soon!

Our big news this month is our collaboration with Google Cloud. With it, we’ve introduced a number of new cloud security and compliance controls, as well as our software-as-as-service (SaaS) platform that runs on Google Cloud. Read this article for what’s new and for details on how you can take advantage of Sysdg capabilities to tackle the challenges of security for containers, Kubernetes, and cloud services on Google Cloud.

We’re really excited to release a technology preview of our new scanning engine. It is between four and 10 times faster than our previous scanner, but it’s not just about speed. It also has extended vulnerability data, better remediation advice, and is now available as a stand-alone binary, making it a lot more portable. We’ve also improved the user experience (and will continue to do so!). It’s not just about scanning, it’s about making the results easy to consume!

We are also very excited to bring Prometheus monitoring integrations directly into the Sysdig UI. We use PromCat.io to test and select the best exporter for every supported application, contributing to the open-source exporters when needed and committing back to the Prometheus ecosystem. We think that the main entity in Kubernetes is the workload. They can either be Deployment, DaemonSet, StatefulSet, or DeploymentConfig in OpenShift. We show you all the workloads, ordered by clusters and namespaces, and identify which kind of application they are. This way, you can manage Prometheus monitoring integrations more easily. At a glance, you can see which workloads need attention, which are reporting metrics correctly, and easily install exporters for some or all of the workloads.

Prometheus monitoring integrations in Sysdig Monitor

As always, please go review our own Release Notes for more details on product updates, and ping your local Sysdig contact if you have questions about anything covered in here.

Sysdig Secure

New Scanning Engine (Technology Preview)

Sysdig Secure is developing a new scanning engine that will deliver major improvements, additional capabilities, and scanning-centric workflows.

New image scanning engine in Sysdig Secure

The first iteration is available to test and provides:

Much faster scan times: 4x to 10x faster initial image analysis.
Extended vulnerability data, including CVSS scores, vectors containing the full exploitability data, availability of an associated public exploit, etc.
Inline scanner available as a stand-alone binary, no longer requires spawning a container.
Better remediation advice, including ‘Which packages are the worst offenders in my image? Considering all the possible fix versions, which one should I apply?’
Improved, more intuitive user experience, with faster response times.

Important: The new engine is still in the “Preview” phase.

This means:

Not suitable for production. There is no forward compatibility guarantee for the data or configuration (yet).
Testing the new scanning preview will NOT affect any existing scanning workflows that leverage the current scanning backend. It is safe to enable the preview on any account.
We would love your feedback if you want to try it out!

Additional fundamental components are still in development; they will be released in an upcoming version.

To test the new engine, simply enable the flag under Settings >User Profile>Sysdig Labs.

Enabling the new image scanning engine in Sysdig Labs

See New Scanning Engine to download the Inline Scanner binary and begin

Sysdig Monitor

Team Scope for Prometheus Remote Write

Sysdig gives you the ability to determine the granularity of data collected by Prometheus Remote Write, to which team members will have visibility. You can specify what data team members can see by identifying tag/value expressions for the metrics. The drop-down defaults to “is,” but can be changed to “is not,” “in,” “contains,” and so on. Complex policies can be created by clicking drop-down to create a new expression, chaining them as AND conditions.

Creating policies for team access to collected remote write metrics

You can view the saved team scope by hovering on the corresponding team from the User menu.

Detail of the metrics access policy by hovering into the info icon

Enhanced User Experience for Monitor Integrations

The UI for Monitoring Integrations has been enhanced to include guided configuration for exporters.

For each integration, you can use the wizard to specify the required information and you will be provided with a single-line command to install the exporter in your cluster. You no longer have to see the documentation or the given exporter source code to guess the name of the variable to configure the credentials of your database or the SSL certificate in the connection string.
If you want to deploy it through your CI/CD pipeline and cannot run commands directly in production, you also have the option to generate the manifests to upload to your repository. If you prefer a package management approach, you can also use Helm charts for the Monitoring Integrations.
After deploying an exporter, you can see whether it is working on the wizard. Sysdig Monitor automatically detects the metrics arriving in your account and associates them to the workload. This way, it is easy to visually detect the applications that are correctly reporting metrics and those that need attention.

Dashboard Enhancements

Ability to edit dashboard and panel name inline in the Panel Editor.

Editing Dashboard title inline in Sysdig monitor

Ability to add a dashboard template to your favorites.

Highlight of the "Add to favorites" button in the Sysdig Monitor UI

Moved the legacy dashboard templates to the Deprecated section.
Supports RabbitMQ Integration. Configure it using the Monitoring Integrations.
Added new dashboard templates for the following:
- Fargate Usage
- Go applications
- Sysdig Admission Controller
- RabbitMQ Integrations
- Kubernetes Controller Manager
- Kubernetes Scheduler
- Kubernetes HPA (promQL)
- CoreDNS

Sysdig Agents

Sysdig Agent

The latest Sysdig Agent release is 12.0.2. Below is a diff of updates since 11.4.1, which we covered in our last update.

Network Security Communication Works As Expected – In some environments, Sysdig agents could not send any Network Security (Kubernetes Network Policies) communications upon not completing CIDR auto-discovery. This issue has been fixed.
Agent No Longer Crashes in Orchestrated Environments – Fixed a problem related to a race condition in orchestrated environments, such as OpenShift v3, where the agent might crash repeatedly at the agent start.
OpenShift 4 Clusters Able To Retrieve Metadata Without Leases – Fixed an issue where OpenShift clusters would fail to report Kubernetes data when the agent service-account did not have the permission to create leases. With this fix, the Sysdig agent falls back to the previous behavior to retrieve the metadata.
Allow Sysdig Backend to Manage Prometheus Configuration – Allow Sysdig backend to manage Prometheus configuration. For more information, see the following:
- (Limited Availability) Configure Monitoring Integrations
- Monitoring Integration
- Prometheus Monitoring with Sysdig
Agent Console Supports Troubleshooting Prometheus Configuration – The Agent Console now supports troubleshooting Prometheus configuration.
Support for Node Leases – Sysdig agent supports using Kubernetes Lease to control how and when connections are made to the Kubernetes API Server. For more information, see the following: Troubleshooting Node Leases.
Support for Podman Environments – Sysdig agent is supported in Podman environments.
Add Startup Delay to Agent to Kubernetes API Server Connection – Added a delay prior to the agent connecting to the Kubernetes API server. The delay time is set based on the number of nodes in the cluster to prevent overloading the API server. This is to support environments where node leases cannot be used.
Stale Capture Files No Longer Exhaust Local File System – Prevent incomplete and stale capture files from being left behind and thereby avoiding storage consumption for such files.
Honor CPU Quotas – Moved the main dragent process to the default cgroup so that CPU quotas can cover all the agent processes.
Containers Are Detected as Expected – Fixed issue where containers are not detected if SystemdCgroup = true is not enabled in the containerd configuration.
Report Correct Container Metadata – Fixed a problem that caused some container metadata such as the image repository and image tag to be reported incorrectly.
Upgrading from 10.8.0 to 11.3.0 No Longer Fails – Provide a http_proxy configuration option to address connection problems post-OpenSSL upgrade from v11.0 to v11.1.

Sysdig Serverless Agent

The latest Sysdig Serverless Agent release is 2.0.1. Below is a diff of updates since 2.0.0, which we covered in our last update.

Fixed Task Stall Issue – Fixed a memory leak in the Serverless Agent instrumentation that could cause the instrumented task to stall. The problem is more likely to be encountered when a large number of captures are generated in quick succession.
Resolved an Agent Error when Reading File Descriptors – Reduced the log level of a benign warning message to debug.

Sysdig Agent – Helm Chart

The latest release of the Sysdig Agent Helm Chart 1.12.28. Below is a diff of updates since 1.12.8, which we covered in our last update.

Fixed read access to networkpolicy resources.
Added affinity to nodeImageAnalyzer DaemonSet.
Fixed Openshift SCC to allow downward API volumes.
Added priorityclass option for image-analyzer daemonset.
Updated agent to 12.0.2.
In chart version 1.12.13, the default agent container resources were set to small. This was a breaking change because upgrading the agent using this chart from an earlier chart where the default was medium could result in less resources configured for it than required.
Added downward API volume to autodetect agent namespace.
Introduced resource profiles.
Added new resources to ClusterRole to support collection of Kubelet metrics.
Bumped host-analyzer version to 0.1.3.
Added mountPath /sys/kernel/debug for eBPF.
Changed the default agent container resources.
Introduced leaderelection.enable for the agent leader election algorithm.

https://charts.sysdig.com/

Changelog: https://github.com/sysdiglabs/charts/blob/master/charts/sysdig/CHANGELOG.md

Node Analyzer

The latest version of Node Analyzer is 0.1.14. It includes some security fixes since our last update.

Node image analyzer can be installed as part of the Sysdig Agent install: https://docs.sysdig.com/en/scan-running-images.html

Inline Scanning Engine

v2.4.6 is the latest release, which we covered in our last update.

Admission Controller

v3.7.2 is the latest release. Below is a diff of updates since v3.8.0, which we covered in our last update.

Fixed scanning request cancelation
Fixed issues with JSON parsing with API changes on staging
Sped up JSON handling
Fixed errors when parsing exceptions for some customers
Added exceptions support

https://docs.sysdig.com/en/admission-controller.html

SDK, CLI and Tools

Sysdig CLI

v0.7.14 is still the latest release, which we covered in our last update.

https://github.com/sysdiglabs/sysdig-platform-cli/releases/tag/v0.7.14
https://sysdiglabs.github.io/sysdig-platform-cli/

Python SDK

v0.16.3 is still the latest release. Below is a diff of updates since v0.16.2, which we covered in our last update.

Add ScanningAlertsClientV1 with support for Scanning Alerts
- New method sdcclient.SdScanningClient.add_repository_alert
- New method sdcclient.SdScanningClient.add_runtime_alert
- New method sdcclient.SdScanningClient.update_repository_alert
- New method sdcclient.SdScanningClient.update_runtime_alert
- Deprecated method sdcclient.SdScanningClient.add_alert
- Deprecated method sdcclient.SdScanningClient.update_alert
- New sdcclient.secure.scanning.ScanningAlertsClientV1 class that has all methods mapped to sdcclient.SdScanningClient for backwards compatibility reasons. For scanning alert related requests, please migrate your code to use ScanningAlertsClientV1.
Require minimum urllib3 1.26.0 version

https://github.com/sysdiglabs/sysdig-sdk-python/releases/tag/v0.16.3

Terraform Provider

v0.5.24 is the latest release. Below is a diff of updates since v0.5.20, which we covered in our last update.

Added documentation for the sysdig_fargate_workload_agent resource.
Updated resource: sysdig_monitor_notification_channel_webhook has a new additional_headers field to pass additional headers to the webhook endpoint.
Added support of custom headers for Webhook notifications.
New resource: sysdig_monitor_alert_promql with support for PromQL Alerts in monitor.
Updated resource: sysdig_secure_benchmark_task, now supports the schema azure_foundations_bench-1.3.0.
New data source: sysdig_fargate_workload_agent, provides an updated fargate workload definition for Sysdig deployment.
Support for PromQL Alerts.
Added azure to valid benchmark schemas.
Added a data source to wrap Fargate tasks in the Sysdig workload agent.
Added role_name to cloud account resource.

Documentation

Falco VS Code Extension

v0.1.0 is still the latest release.

https://github.com/sysdiglabs/vscode-falco/releases/tag/v0.1.0

Sysdig Cloud Connector

v0.11.1 of the Sysdig Cloud Connector was released, we covered v0.8.5 last month.

Here are some highlights of the differences between these versions:

Exposed time spent on processing events using prometheus metrics.
Added a new ingestor to deploy GCP Threat Detections on a Kubernetes Cluster.
Fixed issues with JSON parsing with API changes on staging.
Included resourceName scope entry if exists in the GCP based events.
First release ready for Azure Threat detections.
Added pipeline for auditing ciem activity.
Added Azure validator errors.
Added origin for GCP scanning.
Added support for Falco rule exceptions.
GCP blueprints mappings and new rules.

Check the full list of changes to get all the details.

Sysdig Secure Inline Scan for Github Actions

v3.2.0 is the latest release, which we covered in our last update.

https://github.com/marketplace/actions/sysdig-secure-inline-scan

Sysdig Secure Jenkins Plugin

v2.1.12 is the latest release. Below is a diff of updates since v2.1.11, which we covered in our last update.

Fix Dockerfile existence check. The file must be checked on the agent, not on the master.

https://plugins.jenkins.io/sysdig-secure/

Promcat.io

New applications added:

Harbor 2.2
Keda

Fixes:

Improved documentation and updated versions supported on Redis

What’s new in Sysdig – October 2021