Skip to content

Commit 4d500ed

Browse files
lentzi90lentzi90
committed
Enable k8s upgrade in slef hosted test
This enables the k8s upgrade that is part of the self hosted test. It was previously disabled since the dynamic installation of kubernetes components on the nodes made it hard to change the version in the middle of a test. Also updated the README tables for what versions are tested.
1 parent 3853737 commit 4d500ed

File tree

13 files changed

+134
-25
lines changed

13 files changed

+134
-25
lines changed

‎Makefile

+2-1
Original file line numberDiff line numberDiff line change
@@ -165,7 +165,8 @@ e2e-templates: $(addprefix $(E2E_NO_ARTIFACT_TEMPLATES_DIR)/, \
165165
cluster-template-multi-network.yaml \
166166
cluster-template-without-lb.yaml \
167167
cluster-template.yaml \
168-
cluster-template-flatcar.yaml)
168+
cluster-template-flatcar.yaml \
169+
cluster-template-k8s-upgrade.yaml)
169170
# Currently no templates that require CI artifacts
170171
# $(addprefix $(E2E_TEMPLATES_DIR)/, add-templates-here.yaml) \
171172

‎README.md

+12-9
Original file line numberDiff line numberDiff line change
@@ -48,25 +48,28 @@ This provider's versions are compatible with the following versions of Cluster A
4848

4949
This provider's versions are able to install and manage the following versions of Kubernetes:
5050

51-
| | v1.22 | v1.23 | v1.24 | v1.25 | v1.26 | v1.27 | v1.28 |
52-
|------------------------------------| ----- | ----- | ----- | ----- | ----- | ----- | ----- |
53-
| OpenStack Provider v1alpha5 (v0.6) | + | ✓ | ✓ | ✓ | + | + | + |
54-
| OpenStack Provider v1alpha6 (v0.7) | + | ✓ | ✓ | ✓ | ✓ | ✓ | + |
55-
| OpenStack Provider v1alpha7 | + | + | + | + | ✓ | ✓ | ✓ |
51+
| | v1.25 | v1.26 | v1.27 | v1.28 |
52+
|------------------------------------| ----- | ----- | ----- | ----- |
53+
| OpenStack Provider v1alpha5 (v0.6) | ✓ | + | + | + |
54+
| OpenStack Provider v1alpha6 (v0.7) | ✓ | ✓ | ✓ | + |
55+
| OpenStack Provider v1alpha7 | + | ✓ | ✓ | ★ |
5656

5757
This provider's versions are able to install Kubernetes to the following versions of OpenStack:
5858

5959
| | Queens | Rocky | Stein | Train | Ussuri | Victoria | Wallaby | Xena | Yoga |
6060
|------------------------------------| ------ | ----- | ----- | ----- | ------ | -------- | ------- | ---- | ---- |
61-
| OpenStack Provider v1alpha5 (v0.6) | + | + | + | + | + | ✓ | ✓ | ✓ | ✓ |
62-
| OpenStack Provider v1alpha6 (v0.7) | + | + | + | + | + | ✓ | ✓ | ✓ | ✓ |
63-
| OpenStack Provider v1alpha7 | | + | + | + | + | ✓ | ✓ | ✓ | ✓ |
61+
| OpenStack Provider v1alpha5 (v0.6) | + | + | + | + | + | ✓ | ✓ | ✓ | ★ |
62+
| OpenStack Provider v1alpha6 (v0.7) | + | + | + | + | + | ✓ | ✓ | ✓ | ★ |
63+
| OpenStack Provider v1alpha7 | | + | + | + | + | ✓ | ✓ | ✓ | ★ |
6464

6565
Test status:
6666

67-
- `✓` tested
67+
- `★` currently testing
68+
- `✓` previously tested
6869
- `+` should work, but we weren't able to test it
6970

71+
Older versions may also work but we have not verified.
72+
7073
Each version of Cluster API for OpenStack will attempt to support two Kubernetes versions.
7174

7275
**NOTE:** As the versioning for this project is tied to the versioning of Cluster API, future modifications to this

‎hack/ci/cloud-init/controller.yaml.tpl

+1
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,7 @@
6262
# https://docs.openstack.org/glance/latest/admin/quotas.html
6363
IMAGE_URLS="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/amphora/2022-12-05/amphora-x64-haproxy.qcow2,"
6464
IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/cirros/2022-12-05/cirros-0.6.1-x86_64-disk.img,"
65+
IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/ubuntu/2023-09-29/ubuntu-2204-kube-v1.27.2.img,"
6566
IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/ubuntu/2023-09-29/ubuntu-2204-kube-v1.28.2.img,"
6667
IMAGE_URLS+="https://storage.googleapis.com/artifacts.k8s-staging-capi-openstack.appspot.com/test/flatcar/flatcar-stable-3602.2.0-kube-v1.28.2.img"
6768

‎test/e2e/data/e2e_conf.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,12 @@ variables:
165165
# used to ensure we deploy to the correct management cluster
166166
KUBE_CONTEXT: "kind-capo-e2e"
167167
KUBERNETES_VERSION: "v1.28.2"
168+
KUBERNETES_VERSION_UPGRADE_FROM: "v1.27.2"
169+
KUBERNETES_VERSION_UPGRADE_TO: "v1.28.2"
170+
ETCD_VERSION_UPGRADE_TO: "3.5.9-0"
171+
COREDNS_VERSION_UPGRADE_TO: "v1.10.1"
172+
CONTROL_PLANE_MACHINE_TEMPLATE_UPGRADE_TO: "upgrade-to-control-plane"
173+
WORKERS_MACHINE_TEMPLATE_UPGRADE_TO: "upgrade-to-md-0"
168174
CNI: "../../data/cni/calico.yaml"
169175
CCM: "../../data/ccm/cloud-controller-manager.yaml"
170176
EXP_CLUSTER_RESOURCE_SET: "true"
@@ -179,6 +185,7 @@ variables:
179185
OPENSTACK_FAILURE_DOMAIN: "testaz1"
180186
OPENSTACK_FAILURE_DOMAIN_ALT: "testaz2"
181187
OPENSTACK_IMAGE_NAME: "ubuntu-2204-kube-v1.28.2"
188+
OPENSTACK_IMAGE_NAME_UPGRADE_FROM: "ubuntu-2204-kube-v1.27.2"
182189
OPENSTACK_NODE_MACHINE_FLAVOR: "m1.small"
183190
OPENSTACK_SSH_KEY_NAME: "cluster-api-provider-openstack-sigs-k8s-io"
184191
OPENSTACK_EXTERNAL_NETWORK_ID: ""

‎test/e2e/data/kustomize/common-patches/ccm/kustomization.yaml

+1-3
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,6 @@ kind: Component
66
resources:
77
- ccm.yaml
88

9-
patchesStrategicMerge:
10-
- patch-ccm.yaml
11-
129
patches:
1310
- target:
1411
kind: OpenStackCluster
@@ -18,3 +15,4 @@ patches:
1815
kind: KubeadmControlPlane
1916
name: \${CLUSTER_NAME}-control-plane
2017
path: patch-ccm-cloud-config.yaml
18+
- path: patch-ccm.yaml

‎test/e2e/data/kustomize/common-patches/cni/kustomization.yaml

+1-3
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,9 @@ kind: Component
66
resources:
77
- cni.yaml
88

9-
patchesStrategicMerge:
10-
- patch-cni.yaml
11-
129
patches:
1310
- target:
1411
kind: OpenStackCluster
1512
name: \${CLUSTER_NAME}
1613
path: patch-cluster.yaml
14+
- path: patch-cni.yaml

‎test/e2e/data/kustomize/k8s-upgrade/kcp-patch.yaml

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
2+
kind: KubeadmControlPlane
3+
metadata:
4+
name: ${CLUSTER_NAME}-control-plane
5+
spec:
6+
machineTemplate:
7+
infrastructureRef:
8+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
9+
kind: OpenStackMachineTemplate
10+
name: ${CLUSTER_NAME}-upgrade-from-control-plane

‎test/e2e/data/kustomize/k8s-upgrade/kustomization.yaml

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
---
2+
resources:
3+
- ../default
4+
- upgrade-from-template.yaml
5+
- upgrade-to-template.yaml
6+
7+
patches:
8+
- path: kcp-patch.yaml
9+
- path: md-patch.yaml

‎test/e2e/data/kustomize/k8s-upgrade/md-patch.yaml

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
apiVersion: cluster.x-k8s.io/v1beta1
2+
kind: MachineDeployment
3+
metadata:
4+
name: ${CLUSTER_NAME}-md-0
5+
spec:
6+
template:
7+
spec:
8+
infrastructureRef:
9+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
10+
kind: OpenStackMachineTemplate
11+
name: ${CLUSTER_NAME}-upgrade-from-md-0

‎test/e2e/data/kustomize/k8s-upgrade/upgrade-from-template.yaml

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
3+
kind: OpenStackMachineTemplate
4+
metadata:
5+
name: ${CLUSTER_NAME}-upgrade-from-control-plane
6+
spec:
7+
template:
8+
spec:
9+
cloudName: ${OPENSTACK_CLOUD}
10+
flavor: ${OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR}
11+
identityRef:
12+
kind: Secret
13+
name: ${CLUSTER_NAME}-cloud-config
14+
image: ${OPENSTACK_IMAGE_NAME_UPGRADE_FROM}
15+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
16+
tags:
17+
- control-plane
18+
---
19+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
20+
kind: OpenStackMachineTemplate
21+
metadata:
22+
name: ${CLUSTER_NAME}-upgrade-from-md-0
23+
spec:
24+
template:
25+
spec:
26+
cloudName: ${OPENSTACK_CLOUD}
27+
flavor: ${OPENSTACK_NODE_MACHINE_FLAVOR}
28+
identityRef:
29+
kind: Secret
30+
name: ${CLUSTER_NAME}-cloud-config
31+
image: ${OPENSTACK_IMAGE_NAME_UPGRADE_FROM}
32+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
33+
tags:
34+
- machine

‎test/e2e/data/kustomize/k8s-upgrade/upgrade-to-template.yaml

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# This is just a copy of the normal cluster template OSMTs.
2+
# It is needed because we need a static name that we can put in the e2e config.
3+
# The normal templates has the cluster name as part of the OSMT name.
4+
# See CONTROL_PLANE_MACHINE_TEMPLATE_UPGRADE_TO and WORKERS_MACHINE_TEMPLATE_UPGRADE_TO
5+
# in the e2e config.
6+
---
7+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
8+
kind: OpenStackMachineTemplate
9+
metadata:
10+
name: upgrade-to-control-plane
11+
labels:
12+
clusterctl.cluster.x-k8s.io/move: ""
13+
spec:
14+
template:
15+
spec:
16+
cloudName: ${OPENSTACK_CLOUD}
17+
flavor: ${OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR}
18+
identityRef:
19+
kind: Secret
20+
name: ${CLUSTER_NAME}-cloud-config
21+
image: ${OPENSTACK_IMAGE_NAME}
22+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
23+
tags:
24+
- control-plane
25+
---
26+
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha7
27+
kind: OpenStackMachineTemplate
28+
metadata:
29+
name: upgrade-to-md-0
30+
labels:
31+
clusterctl.cluster.x-k8s.io/move: ""
32+
spec:
33+
template:
34+
spec:
35+
cloudName: ${OPENSTACK_CLOUD}
36+
flavor: ${OPENSTACK_NODE_MACHINE_FLAVOR}
37+
identityRef:
38+
kind: Secret
39+
name: ${CLUSTER_NAME}-cloud-config
40+
image: ${OPENSTACK_IMAGE_NAME}
41+
sshKeyName: ${OPENSTACK_SSH_KEY_NAME}
42+
tags:
43+
- machine

‎test/e2e/shared/defaults.go

+1
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@ const (
5353
FlavorMDRemediation = "md-remediation"
5454
FlavorKCPRemediation = "kcp-remediation"
5555
FlavorFlatcar = "flatcar"
56+
FlavorKubernetesUpgrade = "k8s-upgrade"
5657
)
5758

5859
// DefaultScheme returns the default scheme to use for testing.

‎test/e2e/suites/e2e/self_hosted_test.go

+2-9
Original file line numberDiff line numberDiff line change
@@ -30,23 +30,16 @@ import (
3030

3131
var _ = Describe("When testing Cluster API provider Openstack working on [self-hosted] clusters", func() {
3232
ctx := context.TODO()
33-
shared.SetEnvVar("USE_CI_ARTIFACTS", "true", false)
3433
shared.SetEnvVar("DOWNLOAD_E2E_IMAGE", "true", false)
35-
// TODO(lentzi90): Since we currently rely on USE_CI_ARTIFACTS to get kubernetes set up,
36-
// we are stuck with one version only and cannot do the k8s upgrade that is part of the
37-
// SelfHostedSpec. The reason for this is that the script used to install kubernetes tools comes
38-
// with the KubeadmControlPlane and that does not change when upgrading the k8s version
39-
// (only the machineTemplate changes).
40-
// Until we have a way to work around this, the k8s upgrade test is disabled.
4134
capie2e.SelfHostedSpec(ctx, func() capie2e.SelfHostedSpecInput {
4235
return capie2e.SelfHostedSpecInput{
4336
E2EConfig: e2eCtx.E2EConfig,
4437
ClusterctlConfigPath: e2eCtx.Environment.ClusterctlConfigPath,
4538
BootstrapClusterProxy: e2eCtx.Environment.BootstrapClusterProxy,
4639
ArtifactFolder: e2eCtx.Settings.ArtifactFolder,
47-
SkipUpgrade: true,
40+
SkipUpgrade: false,
4841
SkipCleanup: false,
49-
Flavor: shared.FlavorDefault,
42+
Flavor: shared.FlavorKubernetesUpgrade,
5043
}
5144
})
5245
})

0 commit comments

Comments
 (0)