Skip to content

build: Add AWS clusterclass example #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ repos:
language: system
files: "^docs/"
pass_filenames: false
- id: examples-sync
name: examples-sync
entry: make examples.sync
language: system
files: "^hack/examples/"
pass_filenames: false
- repo: https://github.com/tekwizely/pre-commit-golang
rev: v1.0.0-rc.1
hooks:
Expand Down Expand Up @@ -102,7 +108,7 @@ repos:
name: License headers - YAML and Makefiles
stages: [commit]
files: (^Makefile|\.(ya?ml|mk))$
exclude: ^pkg/handlers/.+/embedded/.+\.ya?ml$
exclude: ^(pkg/handlers/.+/embedded|examples)/.+\.ya?ml$
args:
- --license-filepath
- hack/license-header.txt
Expand Down
48 changes: 32 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,45 +26,61 @@ make SKIP_BUILD=true dev.run-on-kind
To create the ClusterClass and it's Templates with this runtime extension enabled, run:

```shell
kubectl apply --server-side -f examples/capi-quick-start/capd-cluster-class.yaml
kubectl apply --server-side -f examples/capi-quick-start/docker-cluster-class.yaml
```

If creating an AWS cluster using the example files, you will also need to create a secret with your AWS credentials:

```shell
kubectl apply --server-side -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
name: "aws-quick-start-creds"
namespace: capa-system
stringData:
AccessKeyID: ${AWS_ACCESS_KEY_ID}
SecretAccessKey: ${AWS_SECRET_ACCESS_KEY}
SessionToken: ${AWS_SESSION_TOKEN}
EOF
```

To create a cluster, update `clusterConfig` variable and run:

```shell
kubectl apply --server-side -f examples/capi-quick-start/capd-cluster.yaml
kubectl apply --server-side -f examples/capi-quick-start/docker-cluster.yaml
```

Wait until control plane is ready:

```shell
kubectl wait clusters/capd-quick-start --for=condition=ControlPlaneInitialized --timeout=5m
kubectl wait clusters/docker-quick-start --for=condition=ControlPlaneInitialized --timeout=5m
```

To get the kubeconfig for the new cluster, run:

```shell
clusterctl get kubeconfig capd-quick-start > capd-kubeconfig
clusterctl get kubeconfig docker-quick-start > docker-kubeconfig
```

If you are not on Linux, you will also need to fix the generated kubeconfig's `server`, run:

```shell
kubectl config set-cluster capd-quick-start \
--kubeconfig capd-kubeconfig \
--server=https://$(docker port capd-quick-start-lb 6443/tcp)
kubectl config set-cluster docker-quick-start \
--kubeconfig docker-kubeconfig \
--server=https://$(docker port docker-quick-start-lb 6443/tcp)
```

Wait until all nodes are ready (this indicates that CNI has been deployed successfully):

```shell
kubectl --kubeconfig capd-kubeconfig wait nodes --all --for=condition=Ready --timeout=5m
kubectl --kubeconfig docker-kubeconfig wait nodes --all --for=condition=Ready --timeout=5m
```

Show that Calico is running successfully on the workload cluster:

```shell
kubectl --kubeconfig capd-kubeconfig get daemonsets -n calico-system
kubectl --kubeconfig docker-kubeconfig get daemonsets -n calico-system
```

Deploy kube-vip to provide service load-balancer:
Expand All @@ -78,29 +94,29 @@ kubectl create configmap \
--namespace kube-system kubevip \
--from-literal "range-global=${kind_subnet_prefix}100.0-${kind_subnet_prefix}100.20" \
--dry-run=client -oyaml |
kubectl --kubeconfig capd-kubeconfig apply --server-side -n kube-system -f -
kubectl --kubeconfig docker-kubeconfig apply --server-side -n kube-system -f -

helm upgrade kube-vip-cloud-provider kube-vip/kube-vip-cloud-provider --version 0.2.2 \
--install \
--wait --wait-for-jobs \
--namespace kube-system \
--kubeconfig capd-kubeconfig \
--kubeconfig docker-kubeconfig \
--set-string=image.tag=v0.0.6

helm upgrade kube-vip kube-vip/kube-vip --version 0.4.2 \
--install \
--wait --wait-for-jobs \
--namespace kube-system \
--kubeconfig capd-kubeconfig \
--kubeconfig docker-kubeconfig \
--set-string=image.tag=v0.6.0
```

Deploy traefik as a LB service:

```shell
helm --kubeconfig capd-kubeconfig repo add traefik https://helm.traefik.io/traefik
helm --kubeconfig docker-kubeconfig repo add traefik https://helm.traefik.io/traefik
helm repo update &>/dev/null
helm --kubeconfig capd-kubeconfig upgrade --install traefik traefik/traefik \
helm --kubeconfig docker-kubeconfig upgrade --install traefik traefik/traefik \
--version v10.9.1 \
--wait --wait-for-jobs \
--set ports.web.hostPort=80 \
Expand All @@ -111,13 +127,13 @@ helm --kubeconfig capd-kubeconfig upgrade --install traefik traefik/traefik \
Watch for traefik LB service to get an external address:

```shell
watch -n 0.5 kubectl --kubeconfig capd-kubeconfig get service/traefik
watch -n 0.5 kubectl --kubeconfig docker-kubeconfig get service/traefik
```

To delete the workload cluster, run:

```shell
kubectl delete cluster capd-quick-start
kubectl delete cluster docker-quick-start
```

Notice that the traefik service is deleted before the cluster is actually finally deleted.
Expand Down
1 change: 1 addition & 0 deletions devbox.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"actionlint@latest",
"[email protected]",
"clusterctl@latest",
"coreutils@latest",
"crane@latest",
"envsubst@latest",
"findutils@latest",
Expand Down
6 changes: 6 additions & 0 deletions devbox.lock
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@
"source": "devbox-search",
"version": "1.5.1"
},
"coreutils@latest": {
"last_modified": "2023-09-17T10:54:49Z",
"resolved": "github:NixOS/nixpkgs/5148520bfab61f99fd25fb9ff7bfbb50dad3c9db#coreutils",
"source": "devbox-search",
"version": "9.3"
},
"crane@latest": {
"last_modified": "2023-08-08T03:07:33Z",
"resolved": "github:NixOS/nixpkgs/844ffa82bbe2a2779c86ab3a72ff1b4176cec467#crane",
Expand Down
155 changes: 155 additions & 0 deletions examples/capi-quick-start/aws-cluster-class.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
apiVersion: cluster.x-k8s.io/v1beta1
kind: ClusterClass
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start
spec:
controlPlane:
machineInfrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
name: aws-quick-start-control-plane
ref:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlaneTemplate
name: aws-quick-start-control-plane
infrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSClusterTemplate
name: aws-quick-start
patches:
- external:
discoverVariablesExtension: clusterconfigvars.capi-runtime-extensions
generateExtension: clusterconfigpatch.capi-runtime-extensions
name: cluster-config
- definitions:
- jsonPatches:
- op: add
path: /spec/template/spec/region
valueFrom:
template: |
{{ if .region }}{{ .region }}{{ else }}us-west-2{{ end }}
selector:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSClusterTemplate
matchResources:
infrastructureCluster: true
description: aws region to target for cluster creation
name: region
- definitions:
- jsonPatches:
- op: add
path: /spec/template/spec/identityRef
valueFrom:
template: |
kind: AWSClusterStaticIdentity
name: {{ .builtin.cluster.name }}
selector:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSClusterTemplate
matchResources:
infrastructureCluster: true
description: AWSClusterStaticIdentity identityRef to use when creating the cluster
name: identityRef
variables:
- name: region
schema:
openAPIV3Schema:
default: us-west-2
description: aws region to target for cluster creation
example: us-west-2
type: string
workers:
machineDeployments:
- class: default-worker
template:
bootstrap:
ref:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: aws-quick-start-worker-bootstraptemplate
infrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
name: aws-quick-start-worker-machinetemplate
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSClusterTemplate
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start
spec:
template:
spec: {}
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlaneTemplate
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start-control-plane
spec:
template:
spec:
kubeadmConfigSpec:
clusterConfiguration:
apiServer:
extraArgs:
cloud-provider: aws
controllerManager:
extraArgs:
cloud-provider: aws
initConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: aws
name: '{{ ds.meta_data.local_hostname }}'
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: aws
name: '{{ ds.meta_data.local_hostname }}'
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start-control-plane
spec:
template:
spec:
iamInstanceProfile: control-plane.cluster-api-provider-aws.sigs.k8s.io
instanceType: m5.xlarge
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
kind: AWSMachineTemplate
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start-worker-machinetemplate
spec:
template:
spec:
iamInstanceProfile: nodes.cluster-api-provider-aws.sigs.k8s.io
instanceType: m5.2xlarge
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start-worker-bootstraptemplate
spec:
template:
spec:
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: aws
name: '{{ ds.meta_data.local_hostname }}'
44 changes: 44 additions & 0 deletions examples/capi-quick-start/aws-cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start
spec:
clusterNetwork:
pods:
cidrBlocks:
- 192.168.0.0/16
serviceDomain: cluster.local
services:
cidrBlocks:
- 10.128.0.0/12
topology:
class: aws-quick-start
controlPlane:
replicas: 1
variables:
- name: clusterConfig
value:
addons:
cni:
provider: calico
nfd: {}
version: v1.27.5
workers:
machineDeployments:
- class: default-worker
name: md-0
replicas: 1
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AWSClusterStaticIdentity
metadata:
labels:
cluster.x-k8s.io/provider: aws
name: aws-quick-start
spec:
allowedNamespaces:
list:
- default
secretRef: aws-quick-start-creds
Loading