Amazon EKS Security: 5 Hacks and Tricks
by Gilad David Maayan
Amazon Elastic Kubernetes Service (EKS) enables you to automate Kubernetes deployments. When using EKS, you gain access to many AWS services. For example, you can natively integrate with Amazon Elastic Container Registry (ECR), Elastic Load Balancing, AWS Identity and Access Management (IAM), and Amazon Virtual Private Cloud (VPC). However useful, EKS does not cover all security aspects. AWS is responsible for the security of the cloud, but AWS users are responsible for security in the cloud.
What Is Amazon EKS?
Amazon Elastic Kubernetes Service (EKS) is a service that provides a managed Kubernetes control plane. It can help you automate and simplify Kubernetes deployments.
EKS enables you to automatically detect and replace failed control plan instances. You can also use it to run Kubernetes control plane instances across multiple Availability Zones to ensure high availability.
EKS provides automated upgrades and patching for your versions. It uses the most up-to-date version of Kubernetes and grants access to any tooling or plugins available with self managed deployments. Any applications deployed in EKS are compatible with standard Kubernetes environments. This means you can migrate applications easily with no code modifications required.
Amazon EKS integrates with numerous AWS services to enable security and scalability for your cloud applications. These services include:
- Amazon Elastic Container Registry (ECR)
- Elastic Load Balancing
- AWS Identity and Access Management (IAM)
- Amazon Virtual Private Cloud (VPC)
Kubernetes Security Challenges
The biggest security challenges related to Kubernetes are the distributed and ephemeral nature of containers. In exchange for flexibility and availability, Kubernetes requires you to manage the complex security related to dynamic resources and significant attack surface.
Securing Kubernetes requires automated monitoring with service discovery capabilities. It also requires continuous management and logging of traffic into and between containers, pods, and nodes. Additionally, security tooling must account for the wide variety of dependencies that Kubernetes deployments rely on. For example, runtime orchestration tools, integrations, and hosted applications.
Here are critical security threats most Kubernetes deployments are exposed to:
- Container compromise—misconfigured or vulnerable applications enable attackers to gain access to containers. Once inside, attackers can exploit weaknesses in network connections, process controls, or file systems.
- Unauthorized connections—attackers can use compromised containers to connect to other pods or hosts. These connections can be used to laterally traverse networks or to spread malicious code.
- Data exfiltration—attackers may use reverse shells or other malware to tunnel through networks or connect to control servers. This grants the access needed to access and exfiltrate sensitive data.
5 AWS EKS Security Hacks
Ensuring EKS security in your implementation requires applying some best practices and policies during your deployment. The following practices can help you address some of the most common Kubernetes deployment vulnerabilities.
- Treat your infrastructure as immutable and automate the replacement of nodes
Kubernetes is designed to work with immutable resources. When you need to perform updates, new machine images are created and new workers are initialized to replace the old. This can be accomplished in several ways.
One way is to add your new, updated instances to an autoscaling group. As instances are added, your out of date instances can be isolated and drained. This enables you to swap over to the updated image without losing service. The other way is to create a new node group and to transfer traffic from the old to the new. EKS nodes are based on the second approach.
If you want to manually manage your upgrades, you should do so with the help of automation. This helps you ensure that old instances are not overlooked and can make the transfer of traffic smoother.
- Deploy workers onto private subnets
When you deploy workers, you should configure instances to deploy to private subnets. This minimizes worker exposure to the external (Internet) connections and reduces the chance of outside attack.
As of April 2020, EKS began assigning public IP addresses to node groups according to the subnet the groups resided in. Any groups created before then were automatically assigned a public IP. By deploying to private subnets by default you can help ensure that public IPs are not assigned without your intent.
- Do not allow privilege escalation
Privilege escalation can enable processes to alter the security context that the process is run in. This is accomplished with tools like sudo, SUID, and SGID. This functionality is often used by administrators to execute files or processes without restriction.
While privilege escalation is necessary for many administrative tasks, it can pose a threat if standard or illegitimate users or processes are allowed access to it. To prevent your containers from allowing privilege escalation you can configure the allowPriviledgedEscalation property to false or set securityContext.allowPrivilegedEscalation in your podSpec.
- Use AWS KMS for encryption of secrets
AWS Key Management Service (KMS) is a service you can use to encrypt credentials with a unique data encryption key (DEK). You can use KMS to rotate this key on a recurring schedule and to avoid the hard coding of credentials or secrets in your applications and configurations.
To use KMS, you need to use the KMS plugin for Kubernetes. This plugin enables you to store your secrets in etcd as ciphertext rather than plaintext. These secrets can then be decrypted as needed by your Kubernetes API server.
- Log network traffic metadata
AWS VPC Flow Logs enable you to capture metadata related to the traffic flowing through your VPC. This includes source and destination IP addresses, ports used, and accepted or dropped packets. However, this only works well for static IP addresses. To accurately capture information on dynamic IPs you should use tools to enable pod labeling or the addition of metadata.
With the data gathered from your logs, you can analyze communications between your pods and identify suspicious or unusual events. Incorporating metadata or labels enables you to evaluate these logs over time and determine ongoing trends or events. This can help you spot long running attacks and provide information useful for performance optimization.
Amazon EKS is a powerful resource that can help you automate Kubernetes resources. However, this automation does not include security. To ensure the security of your EKS operations, you need to implement security. As a start, you can automate the replacement of nodes, to ensure your EKS operations remain up-to-date.
To reduce the chance of outside attacks, you can deploy workers onto private subnets. In addition, you should never allow privilege escalation, because this will enable processes to alter security context. To protect data and ensure continual health, you can use AWS KMS for encryption of secrets, and log network traffic metadata.
About The Author
Gilad David Maayan is a technology writer who has worked with over 150 technology companies including SAP, Imperva, Samsung NEXT, NetApp and Ixia, producing technical and thought leadership content that elucidates technical solutions for developers and IT leadership. Today he heads Agile SEO, the leading marketing agency in the technology industry.