Skip to content

Commit f0912e3

Browse files
dkoshkinjimmidyson
dkoshkin
and
jimmidyson
authored
feat: set default CoreDNS version (#959)
**What problem does this PR solve?**: Automatically upgrades the CoreDNS version. This is done by always setting `dns.imageTag` in KCP in the CoreDNS handler, based on the mapping to the cluster's Kubernetes version. This component is different from etcd and kube-proxy that is also installed by kubeadm for a few different reasons. - an etcd upgrade is handled by kubeadm - kube-proxy, another "addon", is [upgraded by CAPI core](https://github.com/kubernetes-sigs/cluster-api/blob/75c986db9e38190a2313eaf6e5f97d955fa96b65/controlplane/kubeadm/internal/controllers/controller.go#L476-L480) because its version will match the Kubernetes version [This functiona call](https://github.com/kubernetes-sigs/cluster-api/blob/75c986db9e38190a2313eaf6e5f97d955fa96b65/controlplane/kubeadm/internal/controllers/controller.go#L482-L485) is misleading and [will only update the version if its set in KCP](https://github.com/kubernetes-sigs/cluster-api/blob/6d7104deacddf540c82734ae8abaf309f2ab3b90/controlplane/kubeadm/internal/workload_cluster_coredns.go#L223-L226). To not cause a rollout of all managed clusters by changing the defaults, this PR introduces a new API to opt in. To enable this functionality a client can set this new API like so for new clusters and during cluster upgrades: ``` spec: topology: variables: - name: clusterConfig value: dns: coreDNS: {} ``` **Which issue(s) this PR fixes**: Fixes # **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> **Special notes for your reviewer**: <!-- Use this to provide any additional information to the reviewers. This may include: - Best way to review the PR. - Where the author wants the most review attention on. - etc. --> --------- Co-authored-by: Jimmi Dyson <[email protected]>
1 parent 01ca5b4 commit f0912e3

27 files changed

+486
-92
lines changed

‎api/v1alpha1/clusterconfig_types.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,8 @@ type DNS struct {
322322

323323
type CoreDNS struct {
324324
// Image required for overriding Kubernetes DNS image details.
325+
// If the image version is not specified,
326+
// the default version based on the cluster's Kubernetes version will be used.
325327
// +kubebuilder:validation:Optional
326328
Image *Image `json:"image,omitempty"`
327329
}

‎api/v1alpha1/crds/caren.nutanix.com_awsclusterconfigs.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -379,7 +379,10 @@ spec:
379379
description: CoreDNS defines the CoreDNS configuration for the cluster.
380380
properties:
381381
image:
382-
description: Image required for overriding Kubernetes DNS image details.
382+
description: |-
383+
Image required for overriding Kubernetes DNS image details.
384+
If the image version is not specified,
385+
the default version based on the cluster's Kubernetes version will be used.
383386
properties:
384387
repository:
385388
description: Repository is used to override the image repository to pull from.

‎api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -296,7 +296,10 @@ spec:
296296
description: CoreDNS defines the CoreDNS configuration for the cluster.
297297
properties:
298298
image:
299-
description: Image required for overriding Kubernetes DNS image details.
299+
description: |-
300+
Image required for overriding Kubernetes DNS image details.
301+
If the image version is not specified,
302+
the default version based on the cluster's Kubernetes version will be used.
300303
properties:
301304
repository:
302305
description: Repository is used to override the image repository to pull from.

‎api/v1alpha1/crds/caren.nutanix.com_genericclusterconfigs.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,8 +58,10 @@ spec:
5858
cluster.
5959
properties:
6060
image:
61-
description: Image required for overriding Kubernetes DNS
62-
image details.
61+
description: |-
62+
Image required for overriding Kubernetes DNS image details.
63+
If the image version is not specified,
64+
the default version based on the cluster's Kubernetes version will be used.
6365
properties:
6466
repository:
6567
description: Repository is used to override the image

‎api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -449,7 +449,10 @@ spec:
449449
description: CoreDNS defines the CoreDNS configuration for the cluster.
450450
properties:
451451
image:
452-
description: Image required for overriding Kubernetes DNS image details.
452+
description: |-
453+
Image required for overriding Kubernetes DNS image details.
454+
If the image version is not specified,
455+
the default version based on the cluster's Kubernetes version will be used.
453456
properties:
454457
repository:
455458
description: Repository is used to override the image repository to pull from.

‎docs/content/customization/generic/dns.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,35 @@ If the `dns.coreDNS` property is not specified, then the customization will be s
1515

1616
### Example
1717

18+
The CoreDNS version can be updated automatically. To do this, set `coreDNS` to an empty object:
19+
20+
```yaml
21+
apiVersion: cluster.x-k8s.io/v1beta1
22+
kind: Cluster
23+
metadata:
24+
name: <NAME>
25+
spec:
26+
topology:
27+
variables:
28+
- name: clusterConfig
29+
value:
30+
dns:
31+
coreDNS: {}
32+
```
33+
34+
Applying this configuration will result in the following value being set,
35+
with the version of the CoreDNS image being set based on the cluster's Kubernetes version:
36+
37+
- `KubeadmControlPlaneTemplate`:
38+
39+
- ```yaml
40+
spec:
41+
kubeadmConfigSpec:
42+
clusterConfiguration:
43+
dns:
44+
imageTag: "v1.11.3"
45+
```
46+
1847
To change the repository and tag for the container image for the CoreDNS pod, specify the following configuration:
1948

2049
> Note do not include "coredns" in the repository, kubeadm already appends it.

‎examples/capi-quick-start/aws-cluster-calico-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ spec:
5050
baseOS: ${AMI_LOOKUP_BASEOS}
5151
format: ${AMI_LOOKUP_FORMAT}
5252
org: "${AMI_LOOKUP_ORG}"
53+
dns:
54+
coreDNS: {}
5355
encryptionAtRest:
5456
providers:
5557
- aescbc: {}

‎examples/capi-quick-start/aws-cluster-calico-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ spec:
4444
baseOS: ${AMI_LOOKUP_BASEOS}
4545
format: ${AMI_LOOKUP_FORMAT}
4646
org: "${AMI_LOOKUP_ORG}"
47+
dns:
48+
coreDNS: {}
4749
encryptionAtRest:
4850
providers:
4951
- aescbc: {}

‎examples/capi-quick-start/aws-cluster-cilium-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ spec:
5050
baseOS: ${AMI_LOOKUP_BASEOS}
5151
format: ${AMI_LOOKUP_FORMAT}
5252
org: "${AMI_LOOKUP_ORG}"
53+
dns:
54+
coreDNS: {}
5355
encryptionAtRest:
5456
providers:
5557
- aescbc: {}

‎examples/capi-quick-start/aws-cluster-cilium-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ spec:
4444
baseOS: ${AMI_LOOKUP_BASEOS}
4545
format: ${AMI_LOOKUP_FORMAT}
4646
org: "${AMI_LOOKUP_ORG}"
47+
dns:
48+
coreDNS: {}
4749
encryptionAtRest:
4850
providers:
4951
- aescbc: {}

‎examples/capi-quick-start/docker-cluster-calico-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ spec:
4848
- end: 198.18.1.30
4949
start: 198.18.1.21
5050
provider: MetalLB
51+
dns:
52+
coreDNS: {}
5153
encryptionAtRest:
5254
providers:
5355
- aescbc: {}

‎examples/capi-quick-start/docker-cluster-calico-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ spec:
4343
- end: 198.18.1.30
4444
start: 198.18.1.21
4545
provider: MetalLB
46+
dns:
47+
coreDNS: {}
4648
encryptionAtRest:
4749
providers:
4850
- aescbc: {}

‎examples/capi-quick-start/docker-cluster-cilium-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,8 @@ spec:
4848
- end: 198.18.1.30
4949
start: 198.18.1.21
5050
provider: MetalLB
51+
dns:
52+
coreDNS: {}
5153
encryptionAtRest:
5254
providers:
5355
- aescbc: {}

‎examples/capi-quick-start/docker-cluster-cilium-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ spec:
4343
- end: 198.18.1.30
4444
start: 198.18.1.21
4545
provider: MetalLB
46+
dns:
47+
coreDNS: {}
4648
encryptionAtRest:
4749
providers:
4850
- aescbc: {}

‎examples/capi-quick-start/nutanix-cluster-calico-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,8 @@ spec:
116116
systemDiskSize: 40Gi
117117
vcpuSockets: 2
118118
vcpusPerSocket: 1
119+
dns:
120+
coreDNS: {}
119121
encryptionAtRest:
120122
providers:
121123
- aescbc: {}

‎examples/capi-quick-start/nutanix-cluster-calico-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,8 @@ spec:
113113
systemDiskSize: 40Gi
114114
vcpuSockets: 2
115115
vcpusPerSocket: 1
116+
dns:
117+
coreDNS: {}
116118
encryptionAtRest:
117119
providers:
118120
- aescbc: {}

‎examples/capi-quick-start/nutanix-cluster-cilium-crs.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,8 @@ spec:
116116
systemDiskSize: 40Gi
117117
vcpuSockets: 2
118118
vcpusPerSocket: 1
119+
dns:
120+
coreDNS: {}
119121
encryptionAtRest:
120122
providers:
121123
- aescbc: {}

‎examples/capi-quick-start/nutanix-cluster-cilium-helm-addon.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,8 @@ spec:
113113
systemDiskSize: 40Gi
114114
vcpuSockets: 2
115115
vcpusPerSocket: 1
116+
dns:
117+
coreDNS: {}
116118
encryptionAtRest:
117119
providers:
118120
- aescbc: {}

‎hack/examples/bases/aws/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,9 @@ patches:
4848
- target:
4949
kind: Cluster
5050
path: ../../../patches/encryption.yaml
51+
- target:
52+
kind: Cluster
53+
path: ../../../patches/coredns.yaml
5154

5255
# Delete the clusterclass-specific resources.
5356
- target:

‎hack/examples/bases/docker/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,9 @@ patches:
4141
- target:
4242
kind: Cluster
4343
path: ../../../patches/encryption.yaml
44+
- target:
45+
kind: Cluster
46+
path: ../../../patches/coredns.yaml
4447

4548
# Deploy ServiceLoadBalancer MetalLB
4649
- target:

‎hack/examples/bases/nutanix/cluster/kustomization.yaml.tmpl

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ patches:
3939
- target:
4040
kind: Cluster
4141
path: ../../../patches/encryption.yaml
42+
- target:
43+
kind: Cluster
44+
path: ../../../patches/coredns.yaml
4245

4346
# Remove Additional Trust Bundle ConfigMap
4447
- target:

‎hack/examples/patches/coredns.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Copyright 2024 Nutanix. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
- op: "add"
5+
path: "/spec/topology/variables/0/value/dns"
6+
value:
7+
coreDNS: {}

‎pkg/handlers/generic/mutation/coredns/inject.go

Lines changed: 38 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ package coredns
55

66
import (
77
"context"
8+
"errors"
89

910
apiextensionsv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1"
1011
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -15,6 +16,7 @@ import (
1516
ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
1617

1718
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/api/v1alpha1"
19+
corednsversions "github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/api/versions"
1820
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/common/pkg/capi/clustertopology/handlers/mutation"
1921
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/common/pkg/capi/clustertopology/patches"
2022
"github.com/nutanix-cloud-native/cluster-api-runtime-extensions-nutanix/common/pkg/capi/clustertopology/patches/selectors"
@@ -26,6 +28,10 @@ const (
2628
VariableName = "coreDNS"
2729
)
2830

31+
var ErrDefaultCoreDNSVersionNotFound = errors.New(
32+
"could not determine default CoreDNS version based on the Kubernetes version",
33+
)
34+
2935
type coreDNSPatchHandler struct {
3036
variableName string
3137
variableFieldPath []string
@@ -53,7 +59,7 @@ func (h *coreDNSPatchHandler) Mutate(
5359
vars map[string]apiextensionsv1.JSON,
5460
holderRef runtimehooksv1.HolderReference,
5561
_ ctrlclient.ObjectKey,
56-
_ mutation.ClusterGetter,
62+
clusterGetter mutation.ClusterGetter,
5763
) error {
5864
log := ctrl.LoggerFrom(ctx).WithValues(
5965
"holderRef", holderRef,
@@ -66,7 +72,7 @@ func (h *coreDNSPatchHandler) Mutate(
6672
)
6773
if err != nil {
6874
if variables.IsNotFoundError(err) {
69-
log.V(5).Info("coreDNSVar variable not defined")
75+
log.V(5).Info("coreDNS variable not defined")
7076
return nil
7177
}
7278
return err
@@ -81,34 +87,52 @@ func (h *coreDNSPatchHandler) Mutate(
8187
coreDNSVar,
8288
)
8389

90+
cluster, err := clusterGetter(ctx)
91+
if err != nil {
92+
log.Error(
93+
err,
94+
"failed to get cluster for CoreDNS mutation handler",
95+
)
96+
return err
97+
}
98+
8499
return patches.MutateIfApplicable(
85100
obj, vars, &holderRef, selectors.ControlPlane(), log,
86101
func(obj *controlplanev1.KubeadmControlPlaneTemplate) error {
87102
log.WithValues(
88103
"patchedObjectKind", obj.GetObjectKind().GroupVersionKind().String(),
89104
"patchedObjectName", ctrlclient.ObjectKeyFromObject(obj),
90-
).Info("setting CoreDNS version if needed")
105+
).Info("setting CoreDNS version")
91106

92107
if obj.Spec.Template.Spec.KubeadmConfigSpec.ClusterConfiguration == nil {
93108
obj.Spec.Template.Spec.KubeadmConfigSpec.ClusterConfiguration = &bootstrapv1.ClusterConfiguration{}
94109
}
95110

96-
if coreDNSVar.Image == nil {
97-
return nil
98-
}
99-
100-
dns := obj.Spec.Template.Spec.KubeadmConfigSpec.ClusterConfiguration.DNS
111+
dns := &obj.Spec.Template.Spec.KubeadmConfigSpec.ClusterConfiguration.DNS
101112

102-
if coreDNSVar.Image.Tag != "" {
103-
dns.ImageTag = coreDNSVar.Image.Tag
113+
// Set the CoreDNS image from the variable if it is defined.
114+
if coreDNSVar.Image != nil {
115+
if coreDNSVar.Image.Tag != "" {
116+
dns.ImageTag = coreDNSVar.Image.Tag
117+
}
118+
if coreDNSVar.Image.Repository != "" {
119+
dns.ImageRepository = coreDNSVar.Image.Repository
120+
}
104121
}
105122

106-
if coreDNSVar.Image.Repository != "" {
107-
dns.ImageRepository = coreDNSVar.Image.Repository
123+
// If the CoreDNS image tag is still not set, set the image tag to the default CoreDNS version based on the
124+
// Kubernetes version.
125+
if dns.ImageTag == "" {
126+
defaultCoreDNSVersion, found := corednsversions.GetCoreDNSVersion(
127+
cluster.Spec.Topology.Version,
128+
)
129+
if !found {
130+
return ErrDefaultCoreDNSVersionNotFound
131+
}
132+
133+
dns.ImageTag = defaultCoreDNSVersion
108134
}
109135

110-
obj.Spec.Template.Spec.KubeadmConfigSpec.ClusterConfiguration.DNS = dns
111-
112136
return nil
113137
})
114138
}

0 commit comments

Comments
 (0)