Skip to content

Commit 64fd897

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #1833 from Nordix/clusterclass-tilt-capo/max
🌱 Enhance Tilt integration with CAPO using a ClusterClass template
2 parents e89ec9c + 9a71402 commit 64fd897

File tree

3 files changed

+237
-1
lines changed

3 files changed

+237
-1
lines changed

‎docs/book/src/development/development.md

Lines changed: 96 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,104 @@ make docker-build docker-push
5050

5151
After generating `infrastructure-components.yaml`, replace the `us.gcr.io/k8s-artifacts-prod/capi-openstack/capi-openstack-controller:v0.3.4` with your image.
5252

53+
## Testing Cluster Creation using the 'dev-test' ClusterClass with Tilt
54+
55+
This guide demonstrates how to create a Kubernetes cluster using ClusterClass, specifically designed for a development environment. It includes initializing a Kind cluster, configuring secrets, and applying ClusterClass to deploy your cluster with Tilt.
56+
57+
### Creating a Kind Cluster
58+
59+
Create a Kind cluster. This process involves the initialization of Cluster API providers for OpenStack.
60+
61+
```bash
62+
kind create cluster
63+
export CLUSTER_TOPOLOGY=true
64+
clusterctl init --infrastructure openstack
65+
```
66+
67+
### Secret Configuration
68+
69+
CAPO needs a clouds.yaml file in order to manage the OpenStack resources needed for the Cluster. This should be supplied as a secret named `${CLUSTER_NAME}-cloud-config`. You can create this secret for example with:
70+
71+
```bash
72+
kubectl create secret generic ${CLUSTER_NAME}-cloud-config --from-file=clouds.yaml
73+
```
74+
75+
You can also make Tilt create it for you when you run `tilt up`, more on that in the next section.
76+
5377
## Developing with Tilt
5478

55-
We have support for using [Tilt](https://tilt.dev/) for rapid iterative development. Please visit the [Cluster API documentation on Tilt](https://cluster-api.sigs.k8s.io/developer/tilt.html) for information on how to set up your development environment.
79+
We have support for using [Tilt](https://tilt.dev/) for rapid iterative development. Please visit the [Cluster API documentation on Tilt](https://cluster-api.sigs.k8s.io/developer/tilt.html) for information on how to set up your development environment.
80+
81+
Default values align with the devstack setup, as documented below. When specifying the `KUBERNETES_VERSION`, ensure an image named `ubuntu-2204-kube-{{ KUBERNETES_VERSION }}` is available in your environment, corresponding to that Kubernetes version.
82+
For using Tilt with ClusterClass, update your `tilt-settings.yaml` file as described:
83+
84+
```yaml
85+
kustomize_substitutions:
86+
CLUSTER_TOPOLOGY: "true"
87+
# Define the name of your cluster (e.g., "dev-test")
88+
CLUSTER_NAME: "<cluster_name>"
89+
# Desired Kubernetes Version for Your Cluster (e.g., "v1.28.5")
90+
KUBERNETES_VERSION: "<kubernetes_version>"
91+
# [Optional] SSH Keypair Name for Instances in OpenStack (Default: "")
92+
OPENSTACK_SSH_KEY_NAME: "<openstack_keypair_name>"
93+
# [Optional] Control Plane Machine Flavor (Default: m1.medium)
94+
OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR: "<openstack_control_plane_machine_flavor>"
95+
# [Optional] Node Machine Flavor (Default: m1.small)
96+
OPENSTACK_NODE_MACHINE_FLAVOR: "<openstack_node_machine_flavor>"
97+
# [Optional] OpenStack Cloud Environment (Default: capo-e2e)
98+
OPENSTACK_CLOUD: "<openstack_cloud>"
99+
100+
additional_kustomizations:
101+
secret_kustomization: /path/to/kustomize/secret/configuration
102+
```
103+
104+
Ensure the specified path (`/path/to/kustomize/secret/configuration`) contains both the `clouds.yaml` file and a `kustomization.yaml` file. The `kustomization.yaml` should define the necessary resources, such as a Kubernetes secret, using the `clouds.yaml` file.
105+
106+
For example, if you want to automatically create a secret named `dev-test-cloud-config` with the content of your `clouds.yaml` every time you do `tilt up`, you could do the following.
107+
108+
Create a folder to hold the kustomization.
109+
We will use `/tmp/capo-dev` as example here.
110+
111+
Add the `clouds.yaml` file that you want to use to the folder.
112+
It could look something like this:
113+
114+
```yaml
115+
clouds:
116+
capo-e2e:
117+
auth:
118+
username: demo
119+
password: secretadmin
120+
# If using application credentials you would have something like this instead:
121+
# auth_type: v3applicationcredential
122+
# application_credential_id: abc123
123+
# application_credential_secret: 456def
124+
user_domain_id: default
125+
auth_url: https://example.com/identity
126+
domain_id: default
127+
project_name: demo
128+
verify: false
129+
region_name: RegionOne
130+
```
131+
132+
Create a kustomization file named `kustomization.yaml` in the same folder:
133+
134+
```yaml
135+
apiVersion: kustomize.config.k8s.io/v1beta1
136+
kind: Kustomization
137+
# Do not add random hash to the end of the secret name
138+
generatorOptions:
139+
disableNameSuffixHash: true
140+
secretGenerator:
141+
- files:
142+
- clouds.yaml
143+
name: dev-test-cloud-config
144+
type: Opaque
145+
```
146+
147+
If you now add `/tmp/capo-dev` to the `additional_kustomizations`, tilt will automatically apply
148+
the secret when you do `tilt up`.
149+
150+
To check that the kustomization produces the desired output, do `kustomize build /tmp/capo-dev`.
56151

57152
## Running E2E tests locally
58153

‎templates/cluster-template-development.yaml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
apiVersion: cluster.x-k8s.io/v1beta1
2+
kind: Cluster
3+
metadata:
4+
name: ${CLUSTER_NAME}
5+
spec:
6+
topology:
7+
class: dev-test
8+
version: ${KUBERNETES_VERSION}
9+
workers:
10+
machineDeployments:
11+
- class: default-worker
12+
name: md-0
13+
replicas: 1

‎templates/clusterclass-dev-test.yaml

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
apiVersion: cluster.x-k8s.io/v1beta1
2+
kind: ClusterClass
3+
metadata:
4+
name: dev-test
5+
spec:
6+
controlPlane:
7+
ref:
8+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
9+
kind: KubeadmControlPlaneTemplate
10+
name: ${CLUSTER_NAME}-control-plane-template
11+
machineInfrastructure:
12+
ref:
13+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
14+
kind: OpenStackMachineTemplate
15+
name: ${CLUSTER_NAME}-control-plane-machine-template
16+
infrastructure:
17+
ref:
18+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
19+
kind: OpenStackClusterTemplate
20+
name: ${CLUSTER_NAME}-openstackcluster-template
21+
workers:
22+
machineDeployments:
23+
- class: default-worker
24+
template:
25+
bootstrap:
26+
ref:
27+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
28+
kind: KubeadmConfigTemplate
29+
name: ${CLUSTER_NAME}-default-worker-bootstraptemplate
30+
infrastructure:
31+
ref:
32+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
33+
kind: OpenStackMachineTemplate
34+
name: ${CLUSTER_NAME}-default-worker-machine-template
35+
---
36+
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
37+
kind: KubeadmConfigTemplate
38+
metadata:
39+
name: ${CLUSTER_NAME}-default-worker-bootstraptemplate
40+
spec:
41+
template:
42+
spec:
43+
files: []
44+
joinConfiguration:
45+
nodeRegistration:
46+
kubeletExtraArgs:
47+
cloud-provider: external
48+
provider-id: openstack:///'{{ instance_id }}'
49+
name: '{{ local_hostname }}'
50+
---
51+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
52+
kind: KubeadmControlPlaneTemplate
53+
metadata:
54+
name: ${CLUSTER_NAME}-control-plane-template
55+
spec:
56+
template:
57+
spec:
58+
kubeadmConfigSpec:
59+
clusterConfiguration:
60+
apiServer:
61+
extraArgs:
62+
cloud-provider: external
63+
controllerManager:
64+
extraArgs:
65+
cloud-provider: external
66+
files: []
67+
initConfiguration:
68+
nodeRegistration:
69+
kubeletExtraArgs:
70+
cloud-provider: external
71+
provider-id: openstack:///'{{ instance_id }}'
72+
name: '{{ local_hostname }}'
73+
joinConfiguration:
74+
nodeRegistration:
75+
kubeletExtraArgs:
76+
cloud-provider: external
77+
provider-id: openstack:///'{{ instance_id }}'
78+
name: '{{ local_hostname }}'
79+
---
80+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
81+
kind: OpenStackClusterTemplate
82+
metadata:
83+
name: ${CLUSTER_NAME}-openstackcluster-template
84+
spec:
85+
template:
86+
spec:
87+
apiServerLoadBalancer:
88+
enabled: true
89+
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
90+
dnsNameservers:
91+
- 8.8.8.8
92+
identityRef:
93+
kind: Secret
94+
name: ${CLUSTER_NAME}-cloud-config
95+
managedSecurityGroups: true
96+
nodeCidr: 10.6.0.0/24
97+
---
98+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
99+
kind: OpenStackMachineTemplate
100+
metadata:
101+
name: ${CLUSTER_NAME}-control-plane-machine-template
102+
spec:
103+
template:
104+
spec:
105+
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
106+
flavor: ${OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR:=m1.medium}
107+
identityRef:
108+
kind: Secret
109+
name: ${CLUSTER_NAME}-cloud-config
110+
image:
111+
name: ubuntu-2204-kube-${KUBERNETES_VERSION}
112+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME:=""}
113+
---
114+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
115+
kind: OpenStackMachineTemplate
116+
metadata:
117+
name: ${CLUSTER_NAME}-default-worker-machine-template
118+
spec:
119+
template:
120+
spec:
121+
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
122+
flavor: ${OPENSTACK_NODE_MACHINE_FLAVOR:=m1.small}
123+
identityRef:
124+
kind: Secret
125+
name: ${CLUSTER_NAME}-cloud-config
126+
image:
127+
name: ubuntu-2204-kube-${KUBERNETES_VERSION}
128+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME:=""}

0 commit comments

Comments
 (0)