Recently, Amazon AWS introduced the new feature EC2 Serial Console for instances using Nitro System. It provides a simple and secure way to perform troubleshooting by establishing a connection to the serial port of an instance.
Even though this feature is useful in case of break glass situations, from a security perspective, it could be used by adversaries to gain access through an unguarded secondary entrance.
In this article, we’ll see how EC2 Serial Console works and why this might be a security issue. Using Sysdig Secure, it’s possible to detect if the feature is going to be enabled in your environment and be alerted as soon as possible.
What is EC2 Serial Console about?
EC2 Serial Console provides one-click, text-based access to an instances’ serial port as though a monitor and keyboard were attached to it. It is a simple and secure way to troubleshoot boot and network connectivity issues.
The feature is available for instances that use Nitro System: a combination of network, hypervisor, and security hardware which AWS has adopted for many, but not all, of its EC2.
As it’s disabled by default, if you want to use the feature, you’ll need to enable it for each specific region.
$ aws ec2 enable-serial-console-access --region us-east-2
{
"SerialConsoleAccessEnabled": true
}
When the feature is enabled in the region, you need to push a user public key to authenticate when opening the session for a specific instance.
aws ec2-instance-connect send-serial-console-ssh-public-key \ --instance-id\ --serial-port 0 \ --ssh-public-key file:// \ --region us-east-2
We can now use ssh to establish the connection with the instance, providing the related private key.
Once the connection is established via Serial Console, we need a second step authentication, providing a valid set of credentials for the instance. To correctly authenticate and start interacting with the machine, the local user must have the password previously set.
Security concerns
From a security perspective, we can see it as a second entrance to your environment which might not be guarded.
The EC2 Serial Console feature has been created with the security in mind, giving some resistance to attackers who want to break in using this route. However, as usual, the human factor can be fatal.
As seen before, to actually start using the console you need the following:
- Enable the feature in the region.
- Push the public key before starting a connection, so that only those who have the related private key can connect to the session created.
- Provide valid credentials for a user which must have a password set.
First, by abusing this feature, an adversary can completely bypass any security group rules in place for the instance, pointing directly to its serial console.
An attacker can push an owned public key and start interacting with the server, which is asking for a valid set of credentials. A poor password set for an active user might be fatal, permitting an attacker to connect to the instance and compromise the system.
In addition, there aren’t any mechanisms in place to avoid brute force attacks, leaving the adversary to potentially provide an infinite set of credentials.
Detecting EC2 Serial Console enabled using CloudTrail
Fortunately, it’s possible to bring attention to this kind of side door and alert security teams whenever the feature is enabled in your environment.
As we know, AWS provides strong capabilities to understand what is going on in the environment. CloudTrail records actions taken by a user, role, or an AWS service from AWS Management Console, AWS Command Line Interface, and AWS SDKs and APIs.
Through CloudTrail, we can see when the user admin enables the feature via Management Console, along with other information.
Using another AWS feature, CloudWatch, it’s possible to create alerts based on CloudTrail events. In this case, let’s create an alert if someone enables the feature.
Combined with the Amazon SNS service, we can get alerts via email to respond quickly to critical operational events captured in CloudTrail.
Detecting EC2 Serial Console using Sysdig Secure for cloud
Using Sysdig Secure, the process comes out-of-the box due to how the Sysdig Cloud Connector taps into your cloud infrastructure. By reading events from audit logs and other sources, and evaluating them against Falco rules, you can detect abnormal behavior and misconfigurations. For example, having EC2 Serial Console enabled when you didn’t intend to.
Using Sysdig Secure for cloud, it’s possible to get alerted right after the configuration has been deployed in the cloud environment. The alert is enriched as usual with other useful information about the region, and which user has actually performed the change, to speed up the investigation.
The following rule can detect when the EC2 Serial Console is enabled in a specific region:
- rule: "EC2 Serial Console Access Enabled"
desc: "Detect EC2 serial Console Access enabled in the account for a specific region"
condition:
jevt.value[/eventName]="EnableSerialConsoleAccess"
and (not jevt.value[/errorCode] exists)
and jevt.value[/responseElements/EnableSerialConsoleAccessResponse/serialConsoleAccessEnabled]="true"
output:
The EC2 serial Console Access has been enabled in the region %jevt.value[/awsRegion].
requesting IP=%jevt.value[/sourceIPAddress], AWS region=%jevt.value[/awsRegion],
requesting user=%jevt.value[/userIdentity/arn])"
priority: "WARNING"
tags:
- "cloud"
- "aws"
source: "aws_cloudtrail"
Note how you can use the jevt.value[] expression in the condition and output fields of a Falco rule to refer to fields on a cloud security event. The output event contains useful information, such as in what AWS region the EC2 serial console was enabled, by which account, and from what IP address.
Implementing these kinds of rules will raise the general awareness regarding what is going on in the environment in a specific moment, bringing attention to the unknown.
Conclusion
The EC2 Serial Console feature is going to be really useful for troubleshooting and in break-glass situations. However, it is also important to avoid leaving unguarded secondary entrances in the environment, which might end up being free entry points for attackers.
By using the new Sysdig Secure for cloud, security teams can create custom security rules and get alerted on what is going on, minimizing the chances to get hacked from unknown misconfigurations in the environment.
Our tools make the investigation of security events easier, providing all the relevant information in the same place, in a centralized timeline, and with a correlation between events.
Try it yourself!
With Sysdig Secure for cloud, you can continuously flag cloud misconfigurations before the bad guys get in, and detect suspicious activity like unusual logins from leaked credentials. This all happens in a single console, making it easier to validate your cloud security posture. And it only takes a few minutes to get started!
Start securing your cloud for free with our Sysdig Free Tier!
The Sysdig Secure DevOps Platform provides the visibility you need to confidently run containers, Kubernetes, and cloud. It’s built on an open-source stack with SaaS delivery, and is radically simple to run and scale.
Request a free trial today!