If you want to learn how to prevent a DDoS attack in your cloud environment by detecting the early signs of compromise associated with this threat, then this article should explain most of the best practices required to secure your cloud infrastructure.
From January through July 2022, Sysdig Threat Research team implemented a global honeynet system that captured numerous breaches through multiple attack vectors. When comparing the attack types between Q4 of 2021 and Q1 of 2022 in the 2022 Sysdig Cloud-Native Threat Report, there was a clear shift away from cryptomining towards Distributed Denial-of-Service (DDoS) activity.
Techniques and methods of DoS attacks in the cloud
Previously, we had discussed how to mitigate DoS attacks in Kubernetes using open source tools, such as Calico and Falco, for DoS prevention and detection of DoS in runtime. In the following article, we aim to discuss ways to mitigate DoS attacks specifically in cloud environments – such as AWS, Azure, and GCP.
The patterns and behaviors of a DDoS attack can be segregated into different layers within the OSI model.
- For instance, Application-Layer attacks are anything at the HTTP/s-level. This is Layer 7, the top of the OSI model. Flooding the HTTP/s application with DNS queries or HTTP GET requests is an easy way to achieve this, since we can detect the Killnet cyber attack.
- Attackers can also cause DOS activity at TCP (Layer 4) or via UDP/ICMP activity (Layer 3). These flood the network and servers until they are incapable of processing any legitimate network traffic. The attacker aims to saturate the network connectivity of the target servers.
- Finally, not only attackers could cause this disruption. You might want to test the robustness of your infrastructure and use tools or services to increase traffic and see how your detection tools behave.
In this blog post, we aim to use CNCF Project Falco to detect the Indicators of Compromise (IoC’s) that lead to a DoS Attack in the Cloud. To achieve this, Falco must plug into the audit logging service of the various cloud providers. Thankfully, Falco provides a dedicated plugin for each cloud data source.
By monitoring the expected network traffic/behavior over time, we can design Falco rules to detect unusual network behavior in runtime and prevent a DDoS attack.
How to prevent brute force DDoS attack
As a starting point, it’s important to ensure that your web servers are protected from brute-force attacks. Attackers will aim to get access to a server or temporarily cause it to be unresponsive. Brute force attack involves trying thousands or potentially millions of passwords until the correct password is found.
On one hand, end-users have to follow the security policies when generating strong passwords so that this type of attack does not succeed. On the other hand, most cloud providers offer tools such as rate limiting to prevent brute force attacks.
- Microsoft Azure Rate Limits [Docs]
- Google Cloud Rate Limits [Docs]
- Amazon AWS Rate Limits [Docs]
- IBM Cloud Rate Limits [Docs]
Detection account takeover fraud
One of the main threat detection solutions is to monitor your application’s login page for unauthorized access to user accounts using compromised credentials. Account takeover is an online illegal activity in which an attacker gains unauthorized access to a person’s account.
In the case of AWS’s WAF technology, it has an Account Takeover Prevention (ATP) feature to detect potentially unauthorized access. This is the clearest IoC that could lead to a DoS attack.
The attacker could gain access to the end user’s account in a number of ways, such as using stolen credentials or guessing the victim’s password through a series of attempts. Since Falco plugs into the cloud audit logging service of each cloud provider (i.e., GCP, AWS, and Azure), we can create Falco rules to detect AWS account logins from an unusual IP, for instance:
- rule: Console Login Success From Untrusted IP desc: Detect a console login success from an untrusted IP address condition: >- aws.eventName="ConsoleLogin" and jevt.value[/responseElements/ConsoleLogin]="Success" and not aws.sourceIP in (trusted_ip_addresses) output: >- Detected a console login success from an untrusted IP address (requesting user=%aws.user, requesting IP=%aws.sourceIP, AWS region=%aws.region, arn=%jevt.value[/userIdentity/arn], user agent=%jevt.value[/userAgent]) priority: info source: aws_cloudtrail
If you know which IP addresses a user “should” login from, you can add those to a trusted_ip_addresses list. That way, if a potentially malicious user gains access to the end-user’s credentials and attempts to access the cloud environment, we will be notified that they access the environment from an unknown IP.
We can take these real-time alerts to apply proactive actions, such as enforcing MFA on the account or temporarily suspending the account until we know whether the users had accessed it legitimately. If a user successfully logs in without MFA, we trigger the following rule:
- rule: Console Login Without MFA desc: Detects a console login without MFA. condition: >- aws.eventName="ConsoleLogin" and not aws.errorCode exists and jevt.value[/userIdentity/type]!="AssumedRole" and jevt.value[/responseElements/ConsoleLogin]="Success" and jevt.value[/additionalEventData/MFAUsed]="No" output: >- Detected a console login without MFA (requesting user=%aws.user, requesting IP=%aws.sourceIP, AWS region=%aws.region) priority: critical source: aws_cloudtrail
If you already enforce MFA within your organization, which is one of our strongest recommendations (adding another layer of security), you are making the correct decision.
However, an insider threat, or advanced threat actors who gained access via MFA spamming, could potentially consider disabling MFA to evade detection when they create new IAM users with full permissions. Detecting when MFA is disabled or deleted for an IAM group will help platform teams prevent insecure logins to your cloud provider.
- rule: Azure Deactivate MFA for User Access desc: > Multi-factor authentication requires an individual to present a minimum of two separate forms of authentication before access is granted. Multi-factor authentication provides additional assurance that the individual attempting to gain access is who they claim to be. With multi-factor authentication, an attacker would need to compromise at least two different authentication mechanisms, increasing the difficulty of compromise and thus reducing the risk. condition: >- jevt.value[/operationName]="Disable Strong Authentication" and jevt.value[/properties/result]="success" output: >- Multi-factor authentication configuration has been disabled for a user (requesting user=%jevt.value[/properties/initiatedBy/user/userPrincipalName], requesting IP=%jevt.value[/properties/initiatedBy/user/ipAddress], target user=%jevt.value[/properties/targetResources/0/userPrincipalName]) priority: warning source: azure_platformlogs
Detecting Access Control List overly-permissive
All cloud providers have a feature similar to an Access Control List (ACL). The ACL allows or denies specific inbound or outbound traffic at the subnet-level (Layer 3 and Layer 4 of the OSI). You can create a custom network ACL for your VPC with rules that are similar to those for your security groups, in order to add an additional layer of security to your VPC.
Unfortunately, some organizations open up these ACL’s to the public internet.
As a result, these overly-permissive ACL’s allow the cloud environment to be probed by external adversaries. Air Gapping the cloud environment will prevent external entities from probing your environment, however, many applications need to be opened to the public internet.
That’s why it’s important to use Falco to detect when ACL’s are created with public access, especially if this is not necessarily required:
- rule: Create a Network ACL Entry Allowing Ingress Open to the World desc: >- Detects Access Control List creation allowing ingress open to the world condition: | aws.eventName="CreateNetworkAclEntry" and not aws.errorCode exists and ( not ( jevt.value[/requestParameters/portRange/from]=80 and jevt.value[/requestParameters/portRange/to]=80 ) and not ( jevt.value[/requestParameters/portRange/from]=443 and jevt.value[/requestParameters/portRange/to]=443 ) and ( jevt.value[/requestParameters/cidrBlock]="0.0.0.0/0" or jevt.value[/requestParameters/ipv6CidrBlock]="::/0" ) and jevt.value[/requestParameters/egress]="false" and jevt.value[/requestParameters/ruleAction]="allow" ) output: >- Detected creation of ACL entry allowing ingress open to the world (requesting user=%aws.user, requesting IP=%aws.sourceIP, AWS region=%aws.region, arn=%jevt.value[/userIdentity/arn], network acl id=%jevt.value[/requestParameters/networkAclId], rule number=%jevt.value[/requestParameters/ruleNumber], from port=%jevt.value[/requestParameters/portRange/from], to port=%jevt.value[/requestParameters/portRange/to]) priority: error source: aws_cloudtrail
ACL’s help prevent state-exhaustion attacks. At the networking-layer (Layer 3), the attacker will try to achieve SYN flooding. A SYN flood is a form of denial-of-service where an attacker rapidly initiates a connection to a server without finalizing the connection. The server then has to spend excessive resources waiting for half-opened connections (as part of the TCP handshake workflow), consuming a considerable amount of resources that are required to make the system unresponsive to legitimate ingress traffic.
SYN flooding will consume the TCP connection state tables present in most network/security devices – such as routers, firewalls, ingress controllers, load balancers, and also application servers like Apache. Locking-down access to a network prevents these kinds of Dyn attacks.
Prevent DDoS attack improving web application firewalls configuration
Configure Application (Layer 7) DDoS protections with Web Application Firewalls (WAF). This can either be achieved with a WAF technology offered by the cloud provider, or through a third-party vendor.
In the case of AWS cloud, they offer their own WAF feature. It monitors the HTTP and HTTPS requests that are forwarded to your L7 resources, and lets you control access to your content based on the characteristics of those requests. It utilizes ACL to enforce rate-based rule limits on the volume of traffic from any single IP address. These are requirements of DDoS protection to your application.
Like the SYN flooding we mentioned in the ACL section above, HTTP/S flooding is a popular attack-method for causing DoS. This is caused by flooding the application-layer with DNS queries, which are composed of HTTP GET or POST activity. The goal is to consume excessive application resources such as memory, CPU, and bandwidth.
From an attacker’s perspective, they need to know where the flaws are in the victim’s infrastructure. Will these requests cause database or application processing delays?
If so, the underlying web service should be held-up with malicious requests, and therefore is unable to deliver to other user’s wanting to use the service. As with any L7-style attack, knowing the difference between malicious traffic and normal expected traffic is essential to mitigating the threat.
In this scenario, you can install Falco on a VM/EC2 instance in your cloud environment to prevent a DDoS attack.
Based on system calls from the host, we can see application-level traffic activity. With Falco, you have the option to define a list of trusted domain names (sysdig.com, github.com, and google.com). A network connection to an IP address that isn’t resolved by any of these domain names will trigger the policy.
- list: trusted_domains items: [sysdig.com, github.com, google.com] - rule: Unexpected outbound network connection desc: Detect outbound connections with destinations not on allowed list condition: > Outbound and not (fd.sip.name in (trusted_domains)) output: Unexpected Outbound Connection (container=%container.name command=%proc.cmdline procpname=%proc.pname connection=%fd.name servername=%fd.sip.name serverip=%fd.sip type=%fd.type typechar=%fd.typechar fdlocal=%fd.lip fdremote=%fd.rip) priority: NOTICE
Filter web traffic
Create WAF rules to filter out web requests based on conditions such as HTTP headers, HTTP body, or customer URI’s.
Likewise, use the ACL rules to filter web traffic based on IP address. Similar to using Falco to detect insecure configuration of ACL’s (open to public internet), you can use Falco rules to detect connections going to C2 servers often associated with botnet activity.
- list: c2_server_ip_list items: [220.127.116.11, 18.104.22.168, 22.214.171.124] - rule: Outbound Connection to C2 Servers desc: Detect outbound connection to command & control servers condition: outbound and fd.sip in (c2_server_ip_list) output: Outbound connection to C2 server (command=%proc.cmdline pid=%proc.pid connection=%fd.name user=%user.name user_loginuid=%user.loginuid container_id=%container.id image=%container.image.repository) priority: WARNING tags: [network]
You can, of course, use whichever threat feed you want. We created the c2_server_ip_list based on the first three IP addresses within the Feodo Tracker C2 IP Blocklist.
The team at Abuse.ch provided a simple UI to filter through these IP’s to better understand how they are used in various attack techniques, such as Trojanized loaders, Ransomware, and Denial of Service.
Similar to detecting suspicious outbound traffic to C2 servers, we also want to detect suspicious inbound traffic to our cloud services:
- rule: AWS Suspicious IP Inbound Request desc: >- Detect inbound requests from known suspicious IP sources, such as TOR exit nodes, to AWS services. condition: >- aws.sourceIP in (ti_anon_ips) and not (aws.eventName="ConsoleLogin" and jevt.value[/responseElements/ConsoleLogin]="Failure") output: >- Suspicious IP Inbound Request (aws.sourceIP=%aws.sourceIP, aws.region=%aws.region, aws.eventName=%aws.eventName, aws.user=%aws.user, userAgent=%jevt.value[/userAgent], errorMessage=%jevt.value[/errorMessage]) priority: warning Tags: [Cloud, AWS] source: aws_cloudtrail
Although you can use the same Falco rule logic to block connections to relay networks, such as Tor, it’s important to note that Tor is not an ideal use-case for conducting DDoS attacks.
Since the goal is to overpower the bandwidth of the victim, they typically send UDP packets since those don’t require handshakes or coordination. At the host and workload-level, we can detect suspicious UDP traffic with Falco that deviates away from standard port 53 traffic:
- rule: Unexpected UDP Traffic desc: UDP traffic not on port 53 (DNS) or other commonly used ports condition: >- (inbound_outbound) and do_unexpected_udp_check and fd.l4proto=udp and not Expected_udp_traffic output: > Unexpected UDP Traffic Seen (user.name=%user.name user.loginuid=%user.loginuid proc.cmdline=%proc.cmdline connection=%fd.name proto=%fd.l4proto evt.type=%evt.type %evt.args container.id=%container.id evt.res=%evt.res proc.pid=%proc.pid proc.cwd=%proc.cwd proc.ppid=%proc.ppid proc.pcmdline=%proc.pcmdline proc.sid=%proc.sid proc.exepath=%proc.exepath user.uid=%user.uid user.loginname=%user.loginname group.gid=%group.gid group.name=%group.name container.name=%container.name image=%container.image.repository) priority: notice source: syscall
But because Tor only transports correctly formed TCP streams, not all IP packets, you cannot send UDP packets over Tor (you can’t do specialized forms of this attack like SYN flooding either).
So, in general, attackers who control enough bandwidth to launch an effective DDoS attack can do it just fine without Tor. That said, Tor is useful for data exfiltration before the DDoS attack as it ensures the attackers maintain some form of anonymity. If you want to know more, visit our article How to detect Tor network connections.
Depending on the cloud provider you are using, they will often plug into their own proprietary threat feeds to determine whether connections are coming from known malicious Command and Control (C2) botnet servers and will offer rules to block out these attacks. However, it’s nice to have open source tools like Falco to detect potentially malicious connections or reconnaissance scripts on the host’s syscall-level.
- rule: Detect reconnaissance scripts desc: This rule detects the use of reconnaissance scripts. condition: evt.type=execve and reconnaissance_scripts output: >- Detected reconnaissance script executing (proc.cmdline=%proc.cmdline container=%container.info evt.type=%evt.type evt.arg.request=%evt.arg.request proc.pid=%proc.pid proc.cwd=%proc.cwd proc.ppid=%proc.ppid proc.pcmdline=%proc.pcmdline proc.sid=%proc.sid proc.exepath=%proc.exepath user.uid=%user.uid user.loginuid=%user.loginuid user.loginname=%user.loginname user.name=%user.name group.gid=%group.gid group.name=%group.name container.id=%container.id container.name=%container.name image=%container.image.repository) priority: warning source: syscall
All L7 WAF technologies should offer some form of API security, which can be incorporated into the development and design process of your cloud environment.
API’s are ideal for developers since they packaged all of the relevant commands, payload, and data to produce user interactions. Unfortunately, all of this context and interactivity creates the security risk. As organizations continue to ramp-up their usage of API’s, the unknown attack surface increases.
Many company’s fail to shadow their API’s, and many lack the knowledge of how to hide their API’s as well as confirming whether or not certain API’s are deprecated. Without knowing which ones are third-party vs. no-longer supported, we run the risk of leaving many of these API’s unprotected.
Unfortunately, WAF’s do not address the visibility inventory tracking, risk assessment and threat prevention requirements necessary to protect API’s.
In Falco, we can detect when an API key is created, which helps with overall auditing to confirm who it was made by, when it was made, and to which project the key ID is associated with.
- rule: GCP Create API Keys for a Project desc: Detects the creation of API keys for a project. condition: >- gcp.serviceName="apikeys.googleapis.com" and gcp.methodName endswith ".ApiKeys.CreateApiKey" output: >- An API key for a project has been created (requesting user=%gcp.user, requesting IP=%gcp.callerIP, project id=%jevt.value[/protoPayload/request/projectId], key id=%jevt.value[/protoPayload/response/keyId]) priority: warning source: gcp_auditlog append: false exceptions: 
With respect to cloud-native containerized workloads running in managed cloud Kubernetes services such as GKE, EKS, and AKS, Falco can detect when containers can communicate with Kubernetes API server. This way, we can detect any unusual/unexpected DoS activity at the K8s-level.
- rule: Contact K8S API Server From Container desc: Detect attempts to contact the K8S API Server from a container condition: > evt.type=connect and evt.dir=< and (fd.typechar=4 or fd.typechar=6) and container and not k8s_containers and k8s_api_server and not user_known_contact_k8s_api_server_activities output: Unexpected connection to K8s API Server from container (command=%proc.cmdline pid=%proc.pid %container.info image=%container.image.repository:%container.image.tag connection=%fd.name) priority: NOTICE tags: [network, k8s, container, mitre_discovery]
Detecting unusual DoS activity or preventing a DDoS attack on an API-level in Kubernetes has been made even easier with Kubeshark.
The Kubeshark utility provides deep visibility and monitoring of all API traffic and payloads going in, out, and across containers and pods inside a Kubernetes cluster. These cloud-native workloads often extend to the cloud services that they reside on.
Kubeshark helps by providing auto-generated API and service inventory, which is inferred from the API traffic. That way, we can monitor all API traffic and payloads (whether live or historical) to find API drift and API anomalies. We can trace these down to the source.
Whether using Kubernetes or not, all organizations in the cloud need to automatically track POST/GET activity and to legitimate/illegitimate endpoints or destination addresses. We recommend performing continuous API discovery and visibility either through internal API risk assessment, or by using a third-party software as previously mentioned. Does your cloud provider offer API threat detection and can it natively mitigate these threats in real time?
AWS GuardDuty, for instance, will detect DoS activity for DNS, TCP, UDP, UDP on TCP ports, and general unusual protocol activity.
Mitigation strategies to prevent a DDoS attack in the cloud
In summary, we need to apply certain mitigation strategies that apply to both application and network layers that prevent future DDoS attacks in the cloud.
There are several steps you can take to help prevent DDoS attacks in the cloud:
- Configure your network to filter and block traffic from known malicious sources:
Use firewalls and other network security tools.
- Use content delivery networks (CDNs):
CDNs can help distribute traffic across multiple servers, making it more difficult for a DDoS attack to overwhelm your network.
- Monitor your network for unusual traffic patterns:
Regularly monitor your network for unusual traffic patterns, such as a sudden increase in traffic from a particular source, which could indicate a DDoS attack.
- Prevent account takeover in your cloud tenant:
Many cloud providers offer built-in account takeover and mitigation features by default. MFA provides an extra layer of security when it comes to accessing the console with stolen credentials. As mentioned earlier, Rate limiting is a great way to limit how many incorrect passwords can be attempted against your cloud infrastructure.
- Use a cloud-based DDoS protection service:
There are several cloud-based DDoS protection services available that can help absorb and mitigate DDoS attacks before they reach your network.
- Have a plan in place to respond to a DDoS attack:
It’s important to have a plan in place for how to respond to a DDoS attack, including who to contact and what steps to take to mitigate the attack.
By following these steps, you can help protect your cloud-based systems and prevent a DDoS attack.
Cloud providers do a fantastic job of protecting their customers from DDoS attacks. In fact, this year Google Cloud blocked the largest Layer 7 DDoS attack ever recorded at 46 million Requests Per Second (RPS). That said, this security offering is not enabled for all customers by default. The business that was targeted was a licensed end-user of Google’s Cloud Armor solution. The customer was targeted with a series of HTTPS DDoS attacks, which is a strong use-case for WAF solutions like Cloud Armor.
We detect a growth in DDoS attacks that become weapons for national warfare. In order to conduct successful DDoS campaigns in response to a specific political event, cybercriminal groups need to rapidly scale up their botnet infrastructure.
These DDoS attacks may result in civil, social, or economic damage, depending on what is targeted and the attacker’s goal. For example, at the start of the conflict, pro-Russian DDoS attacks were able to disrupt access to Ukrainian financial institutions.
As discussed in the article, there are multiple ways to prevent a DDoS attack in the cloud.
Volumetric Attacks (Layer 3) will continue to be the most common approach for any web-facing application. Network flooding techniques, such as UDP reflection attacks, are ideal since we mentioned there’s no need to wait on a TCP handshake – attackers can spam the application with UDP or ICMP floods.
As time progressed, we have seen an increase in more sophisticated L7 attacks (not so much aimed at HTTP/DNS floods, but rather attacking the API’s).
Since we acknowledge that many WAF technologies lack the visibility required to secure legitimate vs. illegitimate API connections, many connections will go through without any guardrails. Developer errors, lack of best practices, or improper training can lead to vulnerabilities easily exploited by bad actors. API’s often enable high speed communication to back-end systems, making them prime targets for automated attacks and business logic abuse, even when perfectly coded.
Therefore, we hope that it has become clear how to prevent a DDoS attack in the cloud.
Not all DoS attacks are born equal. As discussed, there are newer DDoS attack vectors for attackers to choose, such as API’s, which are now an integral part of SaaS platforms for automation and integrations. Regardless of the size of your organization, it’s important to apply the basic guardrails. Identify potential weaknesses in your systems, and scan them for known security flaws using vulnerability scanners.
Once you’ve found any misconfigurations, take steps to secure them by implementing appropriate countermeasures and threat detection.
Sysdig Secure helps organizations improve their security posture across cloud providers. Sysdig detects vulnerabilities within Infrastructure-as-Code (IaC) templates, as well as in runtime environments. Sysdig recommends least privilege IAM policies to limit the blast radius in the case of compromised user credentials. By plugging into the cloud audit logs of the various cloud providers, Sysdig provides unparalleled visibility into the entire threat landscape of Azure, GCP, and AWS tenants.