Skip to content

Commit e03d112

Browse files
authored
feat(eks-v2-alpha): eks auto mode support (#33373)
### Issue # (if applicable) Address #32364 in aws-eks-v2-alpha. For EKS Auto Mode, all required configs, including `computeConfig`, `kubernetesNetworkConfig`, and `blockStorage` are managed through the `defaultCapacityType` enum. When set to `DefaultCapacityType.AUTOMODE` (which is the default), these configurations are automatically enabled. The `Cluster` construct in aws-eks-v2-alpha enables EKS Auto Mode by default, managing compute resources through node pools instead of creating default capacity or nodegroups. Users can still opt-in to traditional nodegroup management by setting `defaultCapacityType` to `NODEGROUP` or `EC2`. User Experience: ```ts // Default usage - Auto Mode enabled by default new eks.Cluster(this, 'hello-eks', { vpc, version: eks.KubernetesVersion.V1_32, kubectlProviderOptions: { kubectlLayer: new KubectlV32Layer(this, 'kubectl'), }, // Auto Mode is enabled by default, no need to specify anything }); // Explicit Auto Mode configuration new eks.Cluster(this, 'hello-eks', { vpc, version: eks.KubernetesVersion.V1_32, kubectlProviderOptions: { kubectlLayer: new KubectlV32Layer(this, 'kubectl'), }, defaultCapacityType: eks.DefaultCapacityType.AUTOMODE, // Optional, this is default compute: { nodePools: ['system', 'general-purpose'], // Optional, these are default values nodeRole: customRole, // Optional, custom IAM role for nodes } }); ``` ### Update Summary - [x] EKS Auto Mode is the default mode for `Cluster` construct in V2. When enabled: - Automatically manages compute resources through node pools - Enables elastic load balancing in Kubernetes networking - Enables block storage configuration - Will not create `defaultCapacity` as a `NODEGROUP`(major difference from aws-eks module) - [x] Node pools are case-sensitive and must be "system" and/or "general-purpose" - [x] Auto Mode can coexist with manually added node groups for hybrid deployments - [x] Required IAM policies are automatically attached - [x] Restore the `outputConfigCommand` support previously in `aws-eks` module - [x] integration test - [x] unit tests ### Description of how you validated changes On deploy the autoMode enabled cluster using the code above. ```sh % kubectl create deployment nginx --image=nginx % kubectl get events --sort-by='.lastTimestamp' ``` ``` 20m Normal Nominated pod/nginx-5869d7778c-52pzg Pod should schedule on: nodeclaim/general-purpose-87brc 20m Normal Launched nodeclaim/general-purpose-87brc Status condition transitioned, Type: Launched, Status: Unknown -> True, Reason: Launched 20m Normal DisruptionBlocked nodeclaim/general-purpose-87brc Nodeclaim does not have an associated node 19m Normal NodeHasSufficientPID node/i-0322e9d8dd1b95a51 Node i-0322e9d8dd1b95a51 status is now: NodeHasSufficientPID 19m Normal NodeAllocatableEnforced node/i-0322e9d8dd1b95a51 Updated Node Allocatable limit across pods 19m Normal NodeReady node/i-0322e9d8dd1b95a51 Node i-0322e9d8dd1b95a51 status is now: NodeReady 19m Normal Ready node/i-0322e9d8dd1b95a51 Status condition transitioned, Type: Ready, Status: False -> True, Reason: KubeletReady, Message: kubelet is posting ready status 19m Normal Synced node/i-0322e9d8dd1b95a51 Node synced successfully 19m Normal NodeHasNoDiskPressure node/i-0322e9d8dd1b95a51 Node i-0322e9d8dd1b95a51 status is now: NodeHasNoDiskPressure 19m Normal NodeHasSufficientMemory node/i-0322e9d8dd1b95a51 Node i-0322e9d8dd1b95a51 status is now: NodeHasSufficientMemory 19m Warning InvalidDiskCapacity node/i-0322e9d8dd1b95a51 invalid capacity 0 on image filesystem 19m Normal Starting node/i-0322e9d8dd1b95a51 Starting kubelet. 19m Normal Registered nodeclaim/general-purpose-87brc Status condition transitioned, Type: Registered, Status: Unknown -> True, Reason: Registered 19m Normal Ready nodeclaim/general-purpose-87brc Status condition transitioned, Type: Ready, Status: Unknown -> True, Reason: Ready 19m Normal Initialized nodeclaim/general-purpose-87brc Status condition transitioned, Type: Initialized, Status: Unknown -> True, Reason: Initialized 19m Normal RegisteredNode node/i-0322e9d8dd1b95a51 Node i-0322e9d8dd1b95a51 event: Registered Node i-0322e9d8dd1b95a51 in Controller 19m Normal DisruptionBlocked node/i-0322e9d8dd1b95a51 Node is nominated for a pending pod 19m Normal Scheduled pod/nginx-5869d7778c-52pzg Successfully assigned default/nginx-5869d7778c-52pzg to i-0322e9d8dd1b95a51 19m Warning FailedCreatePodSandBox pod/nginx-5869d7778c-52pzg Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "9bd199c61bd9e93437b10a85af3ddc6965888e01bda96706e153b9e9852f67af": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: Error received from AddNetwork gRPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:50051: connect: connection refused" 19m Normal Pulling pod/nginx-5869d7778c-52pzg Pulling image "nginx" 19m Normal Pulled pod/nginx-5869d7778c-52pzg Successfully pulled image "nginx" in 2.307s (2.307s including waiting). Image size: 72188133 bytes. 19m Normal Created pod/nginx-5869d7778c-52pzg Created container: nginx 19m Normal Started pod/nginx-5869d7778c-52pzg Started container nginx ``` verify the nodes and pods ```sh % kubectl get no NAME STATUS ROLES AGE VERSION i-0322e9d8dd1b95a51 Ready <none> 21m v1.32.0-eks-2e66e76 % kubectl get po NAME READY STATUS RESTARTS AGE nginx-5869d7778c-52pzg 1/1 Running 0 90m ``` ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ### References eksctl YAML experience ```yaml # cluster.yaml apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: my-auto-cluster region: us-west-2 autoModeConfig: # defaults to false enabled: true # optional, defaults to [general-purpose, system] # suggested to leave unspecified nodePools: []string # optional, eksctl creates a new role if this is not supplied # and nodePools are present nodeRoleARN: string ``` Terraform experience: ```hcl provider "aws" { region = "us-east-1" } module "eks" { source = "terraform-aws-modules/eks/aws" cluster_name = "eks-auto-mode-cluster" cluster_version = "1.27" vpc_id = "<your-vpc-id>" subnet_ids = ["<subnet-id-1>", "<subnet-id-2>"] cluster_compute_config = { enabled = true node_pools = ["general-purpose"] # Default pool for Auto Mode } bootstrap_self_managed_addons = true } ``` Pulumi experience ```ts import * as pulumi from "@pulumi/pulumi"; import * as aws from "@pulumi/aws"; // Create EKS cluster with Auto Mode enabled const cluster = new aws.eks.Cluster("example", { name: "example", version: "1.31", bootstrapSelfManagedAddons: false, // Required: Must be false for Auto Mode computeConfig: { enabled: true, // Enable Auto Mode compute nodePools: ["general-purpose"], }, kubernetesNetworkConfig: { elasticLoadBalancing: { enabled: true, // Required for Auto Mode }, }, storageConfig: { blockStorage: { enabled: true, // Required for Auto Mode }, }, }); ``` ### Links - https://aws.amazon.com/about-aws/whats-new/2024/12/amazon-eks-auto-mode/ - https://aws.amazon.com/eks/auto-mode/ - https://aws.amazon.com/blogs/aws/streamline-kubernetes-cluster-management-with-new-amazon-eks-auto-mode/ ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 43ae4fc commit e03d112

File tree

109 files changed

+10272
-1656
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

109 files changed

+10272
-1656
lines changed

packages/@aws-cdk/aws-eks-v2-alpha/README.md

+125-2
Original file line numberDiff line numberDiff line change
@@ -100,22 +100,144 @@ new eks.Cluster(this, 'hello-eks', {
100100
});
101101
```
102102

103+
## EKS Auto Mode
104+
105+
[Amazon EKS Auto Mode](https://aws.amazon.com/eks/auto-mode/) extends AWS management of Kubernetes clusters beyond the cluster itself, allowing AWS to set up and manage the infrastructure that enables the smooth operation of your workloads.
106+
107+
### Using Auto Mode
108+
109+
While `aws-eks` uses `DefaultCapacityType.NODEGROUP` by default, `aws-eks-v2` uses `DefaultCapacityType.AUTOMODE` as the default capacity type.
110+
111+
Auto Mode is enabled by default when creating a new cluster without specifying any capacity-related properties:
112+
113+
```ts
114+
// Create EKS cluster with Auto Mode implicitly enabled
115+
const cluster = new eks.Cluster(this, 'EksAutoCluster', {
116+
version: eks.KubernetesVersion.V1_32,
117+
});
118+
```
119+
120+
You can also explicitly enable Auto Mode using `defaultCapacityType`:
121+
122+
```ts
123+
// Create EKS cluster with Auto Mode explicitly enabled
124+
const cluster = new eks.Cluster(this, 'EksAutoCluster', {
125+
version: eks.KubernetesVersion.V1_32,
126+
defaultCapacityType: eks.DefaultCapacityType.AUTOMODE,
127+
});
128+
```
129+
130+
### Node Pools
131+
132+
When Auto Mode is enabled, the cluster comes with two default node pools:
133+
134+
- `system`: For running system components and add-ons
135+
- `general-purpose`: For running your application workloads
136+
137+
These node pools are managed automatically by EKS. You can configure which node pools to enable through the `compute` property:
138+
139+
```ts
140+
const cluster = new eks.Cluster(this, 'EksAutoCluster', {
141+
version: eks.KubernetesVersion.V1_32,
142+
defaultCapacityType: eks.DefaultCapacityType.AUTOMODE,
143+
compute: {
144+
nodePools: ['system', 'general-purpose'],
145+
},
146+
});
147+
```
148+
149+
For more information, see [Create a Node Pool for EKS Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/create-node-pool.html).
150+
151+
### Node Groups as the default capacity type
152+
153+
If you prefer to manage your own node groups instead of using Auto Mode, you can use the traditional node group approach by specifying `defaultCapacityType` as `NODEGROUP`:
154+
155+
```ts
156+
// Create EKS cluster with traditional managed node group
157+
const cluster = new eks.Cluster(this, 'EksCluster', {
158+
version: eks.KubernetesVersion.V1_32,
159+
defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
160+
defaultCapacity: 3, // Number of instances
161+
defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.LARGE),
162+
});
163+
```
164+
165+
You can also create a cluster with no initial capacity and add node groups later:
166+
167+
```ts
168+
const cluster = new eks.Cluster(this, 'EksCluster', {
169+
version: eks.KubernetesVersion.V1_32,
170+
defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
171+
defaultCapacity: 0,
172+
});
173+
174+
// Add node groups as needed
175+
cluster.addNodegroupCapacity('custom-node-group', {
176+
minSize: 1,
177+
maxSize: 3,
178+
instanceTypes: [ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.LARGE)],
179+
});
180+
```
181+
182+
Read [Managed node groups](#managed-node-groups) for more information on how to add node groups to the cluster.
183+
184+
### Mixed with Auto Mode and Node Groups
185+
186+
You can combine Auto Mode with traditional node groups for specific workload requirements:
187+
188+
```ts
189+
const cluster = new eks.Cluster(this, 'Cluster', {
190+
version: eks.KubernetesVersion.V1_32,
191+
defaultCapacityType: eks.DefaultCapacityType.AUTOMODE,
192+
compute: {
193+
nodePools: ['system', 'general-purpose'],
194+
},
195+
});
196+
197+
// Add specialized node group for specific workloads
198+
cluster.addNodegroupCapacity('specialized-workload', {
199+
minSize: 1,
200+
maxSize: 3,
201+
instanceTypes: [ec2.InstanceType.of(ec2.InstanceClass.C5, ec2.InstanceSize.XLARGE)],
202+
labels: {
203+
workload: 'specialized',
204+
},
205+
});
206+
```
207+
208+
### Important Notes
209+
210+
1. Auto Mode and traditional capacity management are mutually exclusive at the default capacity level. You cannot opt in to Auto Mode and specify `defaultCapacity` or `defaultCapacityInstance`.
211+
212+
2. When Auto Mode is enabled:
213+
- The cluster will automatically manage compute resources
214+
- Node pools cannot be modified, only enabled or disabled
215+
- EKS will handle scaling and management of the node pools
216+
217+
3. Auto Mode requires specific IAM permissions. The construct will automatically attach the required managed policies.
218+
103219
### Managed node groups
104220

105221
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
106222
With Amazon EKS managed node groups, you don't need to separately provision or register the Amazon EC2 instances that provide compute capacity to run your Kubernetes applications. You can create, update, or terminate nodes for your cluster with a single operation. Nodes run using the latest Amazon EKS optimized AMIs in your AWS account while node updates and terminations gracefully drain nodes to ensure that your applications stay available.
107223

108224
> For more details visit [Amazon EKS Managed Node Groups](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html).
109225
110-
**Managed Node Groups are the recommended way to allocate cluster capacity.**
226+
By default, when using `DefaultCapacityType.NODEGROUP`, this library will allocate a managed node group with 2 *m5.large* instances (this instance type suits most common use-cases, and is good value for money).
111227

112-
By default, this library will allocate a managed node group with 2 *m5.large* instances (this instance type suits most common use-cases, and is good value for money).
228+
```ts
229+
new eks.Cluster(this, 'HelloEKS', {
230+
version: eks.KubernetesVersion.V1_32,
231+
defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
232+
});
233+
```
113234

114235
At cluster instantiation time, you can customize the number of instances and their type:
115236

116237
```ts
117238
new eks.Cluster(this, 'HelloEKS', {
118239
version: eks.KubernetesVersion.V1_32,
240+
defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
119241
defaultCapacity: 5,
120242
defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.SMALL),
121243
});
@@ -128,6 +250,7 @@ Additional customizations are available post instantiation. To apply them, set t
128250
```ts
129251
const cluster = new eks.Cluster(this, 'HelloEKS', {
130252
version: eks.KubernetesVersion.V1_32,
253+
defaultCapacityType: eks.DefaultCapacityType.NODEGROUP,
131254
defaultCapacity: 0,
132255
});
133256

0 commit comments

Comments
 (0)