Skip to content

Commit adea1ea

Browse files
dkoshkindkoshkin
authored
feat: support setting kubeadm ignorePreflightErrors (#1097)
**What problem does this PR solve?**: Exposes [CAPI's API](https://github.com/kubernetes-sigs/cluster-api/blob/9dddcb7628831bea6e472ea9afb74e0b4595fc5d/bootstrap/kubeadm/api/v1beta2/kubeadm_types.go#L287-L292C2) to configure kubeadm. This enables working around kubernetes/kubernetes#129462 or ignoring any other kubeadm preflight check. I intentionally added it as this to mimic CAPIs APIs under a single handler so that we can expose additional configuration in the future. ``` nodeRegistration: ignorePreflightErrors: - SystemVerification ``` **Which issue(s) this PR fixes**: Fixes # **How Has This Been Tested?**: <!-- Please describe the tests that you ran to verify your changes. Provide output from the tests and any manual steps needed to replicate the tests. --> Tested with a Docker cluster and verified the generated templates. Also unit tests. **Special notes for your reviewer**: <!-- Use this to provide any additional information to the reviewers. This may include: - Best way to review the PR. - Where the author wants the most review attention on. - etc. --> Eventhough in CAPI's API `taints` are also part of `nodeRegistration`, there is more nuance there that deserves its own handler. Keeping `taints` and `nodeRegistration` also allows to change the implementation of taints in the future to a controller based approach that doesn't require a rollout of new Machines.
1 parent 7d263bd commit adea1ea

21 files changed

+694
-0
lines changed

‎api/v1alpha1/crds/caren.nutanix.com_awsclusterconfigs.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -377,6 +377,23 @@ spec:
377377
default: m5.xlarge
378378
type: string
379379
type: object
380+
nodeRegistration:
381+
default: {}
382+
description: NodeRegistration holds fields that relate to registering the new control-plane node to the cluster.
383+
properties:
384+
ignorePreflightErrors:
385+
default:
386+
- SystemVerification
387+
description: |-
388+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
389+
when the current node is registered.
390+
items:
391+
maxLength: 512
392+
minLength: 1
393+
type: string
394+
maxItems: 50
395+
type: array
396+
type: object
380397
taints:
381398
description: Taints specifies the taints the Node API object should be registered with.
382399
items:

‎api/v1alpha1/crds/caren.nutanix.com_awsworkernodeconfigs.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,24 @@ spec:
9090
description: The AWS instance type to use for the cluster Machines.
9191
type: string
9292
type: object
93+
nodeRegistration:
94+
default: {}
95+
description: NodeRegistration holds fields that relate to registering
96+
the new control-plane node to the cluster.
97+
properties:
98+
ignorePreflightErrors:
99+
default:
100+
- SystemVerification
101+
description: |-
102+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
103+
when the current node is registered.
104+
items:
105+
maxLength: 512
106+
minLength: 1
107+
type: string
108+
maxItems: 50
109+
type: array
110+
type: object
93111
taints:
94112
description: Taints specifies the taints the Node API object should
95113
be registered with.

‎api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -306,6 +306,23 @@ spec:
306306
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
307307
type: string
308308
type: object
309+
nodeRegistration:
310+
default: {}
311+
description: NodeRegistration holds fields that relate to registering the new control-plane node to the cluster.
312+
properties:
313+
ignorePreflightErrors:
314+
default:
315+
- SystemVerification
316+
description: |-
317+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
318+
when the current node is registered.
319+
items:
320+
maxLength: 512
321+
minLength: 1
322+
type: string
323+
maxItems: 50
324+
type: array
325+
type: object
309326
taints:
310327
description: Taints specifies the taints the Node API object should be registered with.
311328
items:

‎api/v1alpha1/crds/caren.nutanix.com_dockerworkernodeconfigs.yaml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,24 @@ spec:
6565
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
6666
type: string
6767
type: object
68+
nodeRegistration:
69+
default: {}
70+
description: NodeRegistration holds fields that relate to registering
71+
the new control-plane node to the cluster.
72+
properties:
73+
ignorePreflightErrors:
74+
default:
75+
- SystemVerification
76+
description: |-
77+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
78+
when the current node is registered.
79+
items:
80+
maxLength: 512
81+
minLength: 1
82+
type: string
83+
maxItems: 50
84+
type: array
85+
type: object
6886
taints:
6987
description: Taints specifies the taints the Node API object should
7088
be registered with.

‎api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -299,6 +299,23 @@ spec:
299299
required:
300300
- daysBeforeExpiry
301301
type: object
302+
nodeRegistration:
303+
default: {}
304+
description: NodeRegistration holds fields that relate to registering the new control-plane node to the cluster.
305+
properties:
306+
ignorePreflightErrors:
307+
default:
308+
- SystemVerification
309+
description: |-
310+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
311+
when the current node is registered.
312+
items:
313+
maxLength: 512
314+
minLength: 1
315+
type: string
316+
maxItems: 50
317+
type: array
318+
type: object
302319
nutanix:
303320
properties:
304321
machineDetails:

‎api/v1alpha1/crds/caren.nutanix.com_nutanixworkernodeconfigs.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,23 @@ spec:
4141
spec:
4242
description: NutanixWorkerNodeConfigSpec defines the desired state of NutanixNodeSpec.
4343
properties:
44+
nodeRegistration:
45+
default: {}
46+
description: NodeRegistration holds fields that relate to registering the new control-plane node to the cluster.
47+
properties:
48+
ignorePreflightErrors:
49+
default:
50+
- SystemVerification
51+
description: |-
52+
IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
53+
when the current node is registered.
54+
items:
55+
maxLength: 512
56+
minLength: 1
57+
type: string
58+
maxItems: 50
59+
type: array
60+
type: object
4461
nutanix:
4562
properties:
4663
machineDetails:

‎api/v1alpha1/nodeconfig_types.go

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,11 @@ type GenericNodeSpec struct {
106106
// Taints specifies the taints the Node API object should be registered with.
107107
// +kubebuilder:validation:Optional
108108
Taints []Taint `json:"taints,omitempty"`
109+
110+
// NodeRegistration holds fields that relate to registering the new control-plane node to the cluster.
111+
// +kubebuilder:validation:Optional
112+
// +kubebuilder:default={}
113+
NodeRegistration *NodeRegistrationOptions `json:"nodeRegistration,omitempty"`
109114
}
110115

111116
// The node this Taint is attached to has the "effect" on
@@ -146,6 +151,19 @@ const (
146151
TaintEffectNoExecute TaintEffect = "NoExecute"
147152
)
148153

154+
// NodeRegistrationOptions holds fields that relate to registering a new control-plane or node to the cluster,
155+
// either via "kubeadm init" or "kubeadm join".
156+
type NodeRegistrationOptions struct {
157+
// IgnorePreflightErrors specifies a slice of pre-flight errors to be ignored by kubeadm
158+
// when the current node is registered.
159+
// +kubebuilder:validation:Optional
160+
// +kubebuilder:default={"SystemVerification"}
161+
// +kubebuilder:validation:MaxItems=50
162+
// +kubebuilder:validation:items:MinLength=1
163+
// +kubebuilder:validation:items:MaxLength=512
164+
IgnorePreflightErrors []string `json:"ignorePreflightErrors,omitempty"`
165+
}
166+
149167
//nolint:gochecknoinits // Idiomatic to use init functions to register APIs with scheme.
150168
func init() {
151169
SchemeBuilder.Register(

‎api/v1alpha1/zz_generated.deepcopy.go

Lines changed: 25 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎docs/content/customization/generic/node-registration.md

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
+++
2+
title = "Node registration configuration"
3+
+++
4+
5+
Below is a list of node registration configuration options that can be set for `kubeadm init` and `kubeadm join`.
6+
7+
This customization will be available when the
8+
[provider-specific cluster configuration patch]({{< ref "..">}}) is included in the `ClusterClass`.
9+
10+
## Example
11+
12+
### ignorePreflightErrors
13+
14+
Kubeadm runs preflight checks to ensure the machine is compatible with Kubernetes and its dependencies.
15+
The `SystemVerification` check is known to result in false positives.
16+
For example, it fails when the Linux Kernel version is not supported by kubeadm,
17+
even if the kernel has all the required features.
18+
For this reason, we skip the check by default.
19+
20+
#### Control plane
21+
22+
To configure `ignorePreflightErrors` for the control plane nodes, specify the following configuration:
23+
24+
```yaml
25+
apiVersion: cluster.x-k8s.io/v1beta1
26+
kind: Cluster
27+
metadata:
28+
name: <NAME>
29+
spec:
30+
topology:
31+
variables:
32+
- name: clusterConfig
33+
value:
34+
controlPlane:
35+
nodeRegistration:
36+
ignorePreflightErrors:
37+
- SystemVerification
38+
```
39+
40+
Applying this configuration will result in the following value being set:
41+
42+
- `KubeadmControlPlaneTemplate`:
43+
44+
- ```yaml
45+
spec:
46+
kubeadmConfigSpec:
47+
initConfiguration:
48+
nodeRegistration:
49+
nodeRegistration:
50+
ignorePreflightErrors:
51+
- SystemVerification
52+
joinConfiguration:
53+
nodeRegistration:
54+
ignorePreflightErrors:
55+
- SystemVerification
56+
```
57+
58+
#### Worker node
59+
60+
`ignorePreflightErrors` for individual nodepools can be configured similarly:
61+
62+
```yaml
63+
apiVersion: cluster.x-k8s.io/v1beta1
64+
kind: Cluster
65+
metadata:
66+
name: <NAME>
67+
spec:
68+
topology:
69+
workers:
70+
machineDeployments:
71+
- class: default-worker
72+
name: md-0
73+
variables:
74+
overrides:
75+
- name: workerConfig
76+
value:
77+
nodeRegistration:
78+
ignorePreflightErrors:
79+
- SystemVerification
80+
```
81+
82+
Applying this configuration will result in the following value being set:
83+
84+
- `KubeadmConfigTemplate`:
85+
86+
- ```yaml
87+
spec:
88+
joinConfiguration:
89+
nodeRegistration:
90+
ignorePreflightErrors:
91+
- SystemVerification
92+
```
93+
94+
By default, the following value will be set for both control plane and worker nodes:
95+
96+
```yaml
97+
variables:
98+
- name: clusterConfig
99+
value:
100+
controlPlane:
101+
nodeRegistration:
102+
ignorePreflightErrors:
103+
- SystemVerification
104+
- name: workerConfig
105+
value:
106+
nodeRegistration:
107+
ignorePreflightErrors:
108+
- SystemVerification
109+
```
110+
111+
This can be enabled by setting `ignorePreflightErrors` to an empty list:
112+
113+
```yaml
114+
variables:
115+
- name: clusterConfig
116+
value:
117+
controlPlane:
118+
nodeRegistration:
119+
ignorePreflightErrors: []
120+
- name: workerConfig
121+
value:
122+
nodeRegistration:
123+
ignorePreflightErrors: []
124+
```

‎examples/capi-quick-start/docker-cluster-calico-crs.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ spec:
4949
- end: 198.18.1.30
5050
start: 198.18.1.21
5151
provider: MetalLB
52+
controlPlane: {}
5253
dns:
5354
coreDNS: {}
5455
encryptionAtRest:

‎examples/capi-quick-start/docker-cluster-calico-helm-addon.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ spec:
4444
- end: 198.18.1.30
4545
start: 198.18.1.21
4646
provider: MetalLB
47+
controlPlane: {}
4748
dns:
4849
coreDNS: {}
4950
encryptionAtRest:

‎examples/capi-quick-start/docker-cluster-cilium-crs.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ spec:
4949
- end: 198.18.1.30
5050
start: 198.18.1.21
5151
provider: MetalLB
52+
controlPlane: {}
5253
dns:
5354
coreDNS: {}
5455
encryptionAtRest:

‎examples/capi-quick-start/docker-cluster-cilium-helm-addon.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ spec:
4444
- end: 198.18.1.30
4545
start: 198.18.1.21
4646
provider: MetalLB
47+
controlPlane: {}
4748
dns:
4849
coreDNS: {}
4950
encryptionAtRest:

‎hack/examples/patches/initialize-variables.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
value:
77
- name: "clusterConfig"
88
value:
9+
controlPlane: {}
910
addons:
1011
clusterAutoscaler: {}
1112
nfd: {}

0 commit comments

Comments
 (0)