EKS on EC2 Spot Instances

Project Overview

In this project, I designed and implemented a cost-effective Kubernetes platform using Amazon EKS running on EC2 spot instances. The solution significantly reduced infrastructure costs while maintaining high availability and performance for production workloads.

Technical Challenge

The main challenges included:

• Ensuring workload resilience despite potential spot instance interruptions
• Maintaining consistent cluster performance with heterogeneous instance types
• Optimizing instance selection for cost-effectiveness without compromising performance
• Implementing proper autoscaling that works correctly with spot instances

Solution

I developed a comprehensive solution that included:

• A multi-AZ EKS cluster with a mix of on-demand and spot instances
• Automated instance selection based on CPU/memory requirements and cost efficiency
• Pod disruption budgets and graceful termination handling
• Advanced node group configurations for workload-specific requirements

Implementation Details

Instance Selection Strategy

One of the key aspects of the implementation was selecting appropriate instance types. I used the amazon-ec2-instance-selector tool to identify homogeneous instance types with the same CPU and memory specifications:

ec2-instance-selector --memory 16 --vcpus 8 --cpu-architecture x86_64 -r us-east-1

This approach yielded instance types like c5.2xlarge, c6i.2xlarge, and others with identical resource specifications, which is crucial for proper cluster autoscaler functioning.

For cost analysis, I leveraged the instances.vantage.sh tool to compare pricing across the selected instance types, enabling data-driven decisions about which instances to include in the node groups.

Karpenter Implementation

To overcome the limitations of the standard cluster autoscaler with heterogeneous instance types, I implemented Karpenter as the node provisioning solution. This allowed for:

• Just-in-time node provisioning based on pod requirements
• Automatic selection of the most cost-effective instance types
• Graceful node termination and pod rescheduling during spot interruptions
• Dynamic node consolidation to minimize resource wastage

The Karpenter configuration included custom provisioners for different workload profiles, ensuring that each type of application received the most appropriate instance type.

Spot Instance Handling

To handle spot instance interruptions gracefully, I implemented:

• Pod disruption budgets (PDBs) for all critical workloads
• Lifecycle hooks to capture termination notices
• A custom controller that monitored spot instance termination notices and triggered graceful pod evictions
• Stateful workload protection with appropriate storage configurations

Monitoring and Cost Optimization

The solution included comprehensive monitoring and cost optimization features:

• Real-time spot instance savings tracking
• Automated reporting of cost efficiency metrics
• Alerts for spot market price fluctuations
• Periodic review and adjustment of instance type selection based on pricing trends

Results

The implementation delivered impressive results:

• 70% reduction in compute infrastructure costs
• 99.95% service availability despite spot instance interruptions
• Improved resource utilization by 35%
• Seamless scaling during peak traffic periods

Lessons Learned

Key takeaways from this project include:

• The importance of selecting homogeneous instance types when using the cluster autoscaler
• The advantages of Karpenter over the standard cluster autoscaler for spot instance management
• The need for proper application design to handle node terminations gracefully
• The value of data-driven instance selection based on both performance and cost metrics

This project demonstrated that with proper architecture and tooling, Kubernetes workloads can run reliably on spot instances while achieving significant cost savings.